5 ms·

Quick Redis Benchmark: AWS vs. Azure

25 points by felixrieseberg 13y ago

nsp 13y ago
Color me impressed, 3x throughput for the price(60,000vs20,000) for azure might be enough for me to create an account. Can anyone comment on what accounts for the differential? It doesn't seem like the hardware would do it,but I could be wrong.
- deleted 13y ago
  [deleted]
- bryceb 13y ago
  EC2's virtualization is pretty terrible for Redis http://redis.io/topics/latency http://redis.io/topics/latency The newer m3 instance types are better, but for the most part Redis is scarcely usable on EC2.
  - dvirsky 13y ago
    I wouldn't go as far as calling it scarcely usable, but EC2 does add some overhead to running redis. I've been using it on EC2 for over 2 years (some of the metrics in the doc you linked are on my machines BTW :) ), and you just have to find workarounds for persistence because of the high fork times - it adds a bit of complexity, but a dedicated slave for saving is not that big a deal, and can reside on the same instance as the master). We've recently moved to m3 instances and the problem has completely gone away, hopefully Amazon will migrate all instances to use HVM.
facorreia 13y ago
Another recent benchmark: Cloud Server Performance: A Comparative Analysis of 5 Large Cloud IaaS Providers http://www.cloudspectator.com/cloud-server-performance-a-comparative-analysis-of-5-large-cloud-iaas-providers-3/ http://www.cloudspectator.com/cloud-server-performance-a-com...
- barista 13y ago
  Interesting that Windows Azure comes up at the top not only purely in performance but also in the price performance ratio as well.
  - facorreia 13y ago
    Indeed, they're matching AWS' prices for IaaS, at least on pay-as-you-go. For reserved instances there seems to be a difference.
forgotAgain 13y ago
The AWS medium tested is a single core. The Azure is dual core. Context switching could account for the difference in preformance. Just because they are both labeled medium doesn't mean they're comparable. The Azure is also about %50 more the AWS last time I looked.. Edit: I was wrong about the pricing. They are similarly priced.
- MBCook 13y ago
  They both cost $0.12/hour unreserved, so it's a fair comparison if you are thinking performance per dollar.
  - forgotAgain 13y ago
    http://www.windowsazure.com/en-us/pricing/details/virtual-machines/ http://www.windowsazure.com/en-us/pricing/details/virtual-ma... Azure is listed at $0.18/hour. Edit: My mistake. Windows is $0.18/hour, Linux is $0.12. Edit2: Now that's interesting, Microsoft charges more for a Windows VM.
    - felixrieseberg 13y ago
      It's an honest approach: Windows VMs include a license.
      - cheald 13y ago
        You'd think that Microsoft wouldn't have to license their own software.
        pdubs 13y ago
        It would probably be seen as anti-competitive if they licensed between business units for free.
  - velodrome 13y ago
    Performance/dollar: Azure wins - 3.1:1 on demand Azure wins - 3:1 on 1 year reserved (author is incorrect here - does not account for initial reserve payment) Less noisy neighbors? I am not sure. Seems like a good deal though.
omarqureshi 13y ago
Having used their Linux offerings in 2012 - the only thing that I can say is whilst Azure does provide good performance for the price. You lose out on stability, sanity and on choice. I know the cloud is supposed to be a place where the servers are volatile ... BUT ... random reboots with non-writeable disks when they come back as well as network connectivity problems between servers are not something that I've experienced at Rackspace or AWS. The support at Azure is pretty laughable ("what's a public key?" from one of their so-called TCP/IP experts) if you manage to get a phone call with the folks over at Redmond. Otherwise all you have is forum support where you are basically treated like an idiot and blamed for their mistakes unless you can prove otherwise. The Windows offering is a little better, more stable for sure - but, then you run into crazy firewall issues if you are using something like AzureRunMe to do the work and set-up bespoke web app servers. For what i spent the best part of a month failing at getting set up at Azure, took me a few days to set up at Rackspace, with configuration management and service monitoring. I wont be rushing back to use Azure again any time soon and be warned that this benchmark is only a small glimpse at what is otherwise a mighty shit sandwich.
- tstrimple 13y ago
  To be fair, Azure IaaS was in preview in 2012. It didn't become generally available until April of this year. http://blogs.msdn.com/b/windowsazure/archive/2013/04/16/the-power-of-and.aspx http://blogs.msdn.com/b/windowsazure/archive/2013/04/16/the-...
  - omarqureshi 13y ago
    totally aware of that, that doesn't discount from what was in my opinion truly terrible customer service and knowledge of their own products by the engineers. it left me with a really bad taste in my mouth.
- shaydoc 13y ago
  I can't comment on linux but I've had no issues with Azure using windows, in fact I would say its fantastic. I been have using cloud services web/worker roles, messaging, blob and SQL azure too... I have been also integrating cloud TFS publishing, and automated deployments. Its been a dream so far. I am looking at the elastic scalability block also for scaling out and in, it all looks great. I mean transitioning from staging to production is just a virtual IP swap, sweet... So if you get your setup right and go with the Microsoft cloud setup I think its smooth sailing.
zerop 13y ago
Try linode, numbers are going to surprise you.

nsp 13y ago

Color me impressed, 3x throughput for the price(60,000vs20,000) for azure might be enough for me to create an account. Can anyone comment on what accounts for the differential? It doesn't seem like the hardware would do it,but I could be wrong.

deleted 13y ago
[deleted]
bryceb 13y ago
EC2's virtualization is pretty terrible for Redis http://redis.io/topics/latency http://redis.io/topics/latency The newer m3 instance types are better, but for the most part Redis is scarcely usable on EC2.
- dvirsky 13y ago
  I wouldn't go as far as calling it scarcely usable, but EC2 does add some overhead to running redis. I've been using it on EC2 for over 2 years (some of the metrics in the doc you linked are on my machines BTW :) ), and you just have to find workarounds for persistence because of the high fork times - it adds a bit of complexity, but a dedicated slave for saving is not that big a deal, and can reside on the same instance as the master). We've recently moved to m3 instances and the problem has completely gone away, hopefully Amazon will migrate all instances to use HVM.

facorreia 13y ago

Another recent benchmark: Cloud Server Performance: A Comparative Analysis of 5 Large Cloud IaaS Providers http://www.cloudspectator.com/cloud-server-performance-a-comparative-analysis-of-5-large-cloud-iaas-providers-3/ http://www.cloudspectator.com/cloud-server-performance-a-com...

barista 13y ago
Interesting that Windows Azure comes up at the top not only purely in performance but also in the price performance ratio as well.
- facorreia 13y ago
  Indeed, they're matching AWS' prices for IaaS, at least on pay-as-you-go. For reserved instances there seems to be a difference.

forgotAgain 13y ago

The AWS medium tested is a single core. The Azure is dual core. Context switching could account for the difference in preformance. Just because they are both labeled medium doesn't mean they're comparable. The Azure is also about %50 more the AWS last time I looked.. Edit: I was wrong about the pricing. They are similarly priced.

MBCook 13y ago
They both cost $0.12/hour unreserved, so it's a fair comparison if you are thinking performance per dollar.
- forgotAgain 13y ago
  http://www.windowsazure.com/en-us/pricing/details/virtual-machines/ http://www.windowsazure.com/en-us/pricing/details/virtual-ma... Azure is listed at $0.18/hour. Edit: My mistake. Windows is $0.18/hour, Linux is $0.12. Edit2: Now that's interesting, Microsoft charges more for a Windows VM.
  - felixrieseberg 13y ago
    It's an honest approach: Windows VMs include a license.
    - cheald 13y ago
      You'd think that Microsoft wouldn't have to license their own software.
      - pdubs 13y ago
        It would probably be seen as anti-competitive if they licensed between business units for free.
- velodrome 13y ago
  Performance/dollar: Azure wins - 3.1:1 on demand Azure wins - 3:1 on 1 year reserved (author is incorrect here - does not account for initial reserve payment) Less noisy neighbors? I am not sure. Seems like a good deal though.

omarqureshi 13y ago

Having used their Linux offerings in 2012 - the only thing that I can say is whilst Azure does provide good performance for the price. You lose out on stability, sanity and on choice. I know the cloud is supposed to be a place where the servers are volatile ... BUT ... random reboots with non-writeable disks when they come back as well as network connectivity problems between servers are not something that I've experienced at Rackspace or AWS. The support at Azure is pretty laughable ("what's a public key?" from one of their so-called TCP/IP experts) if you manage to get a phone call with the folks over at Redmond. Otherwise all you have is forum support where you are basically treated like an idiot and blamed for their mistakes unless you can prove otherwise. The Windows offering is a little better, more stable for sure - but, then you run into crazy firewall issues if you are using something like AzureRunMe to do the work and set-up bespoke web app servers. For what i spent the best part of a month failing at getting set up at Azure, took me a few days to set up at Rackspace, with configuration management and service monitoring. I wont be rushing back to use Azure again any time soon and be warned that this benchmark is only a small glimpse at what is otherwise a mighty shit sandwich.

tstrimple 13y ago
To be fair, Azure IaaS was in preview in 2012. It didn't become generally available until April of this year. http://blogs.msdn.com/b/windowsazure/archive/2013/04/16/the-power-of-and.aspx http://blogs.msdn.com/b/windowsazure/archive/2013/04/16/the-...
- omarqureshi 13y ago
  totally aware of that, that doesn't discount from what was in my opinion truly terrible customer service and knowledge of their own products by the engineers. it left me with a really bad taste in my mouth.
shaydoc 13y ago
I can't comment on linux but I've had no issues with Azure using windows, in fact I would say its fantastic. I been have using cloud services web/worker roles, messaging, blob and SQL azure too... I have been also integrating cloud TFS publishing, and automated deployments. Its been a dream so far. I am looking at the elastic scalability block also for scaling out and in, it all looks great. I mean transitioning from staging to production is just a virtual IP swap, sweet... So if you get your setup right and go with the Microsoft cloud setup I think its smooth sailing.

zerop 13y ago

Try linode, numbers are going to surprise you.