4 ms·

Azure overtakes Amazon's cloud in performance test

18 points by chermanowicz 14y ago

meaty 14y ago
That genuinely doesn't surprise me at all. I've used both extensively and Azure definitely has the edge with respect to performance (and management which is pretty neat). Also I've had severe reliability problems with S3 over the last couple of years. Azure support is actually better as well. I've found it very easy to get through to a human unlike when we had a major EC2 problem. However, I'd pick neither of them if I had to choose and would go with a smaller, old fashioned dedicated hosting company as you are rarely lost in the noise then if something goes bang. Prime example being Azure's reporting services which was dead for nearly an entire week.
- rschmitty 14y ago
  What kind of problems have you had with S3? I think we've all been affected by EBS in someway the last couple of years but I've not run into S3 issues. Could you elaborate?
  - meaty 14y ago
    Connection reliability (we upload large files), the .net API is unreliable in the AWS SDK and lots of spurious issues. We've basically had to implement a retry wrapper around their API.
Flenser 14y ago
The first question I always have when I read headlines such as this is "Who paid for it?" I filled in the form for to download the report and it contains the following: Disclaimer The tests reported upon in this document are conducted by us using our own test tools under test conditions chosen by us. The test conditions were chosen by us to reasonably represent what our customers would experience using our Service with their representative environments and workloads. The tests have been designed by us to only look at the performance aspects of the CSPs that we believe are relevant to our customers – it is intentionally narrow in scope. Nasuni is not in the business of benchmarking CSPs, certifying test results or selling performance metrics. We have attempted to make sure the tests are fair and consistent within our selected parameters and have worked with several of the vendors to confirm our results. Our tests are not meant to indicate performance from each CSP under ideal conditions to the CSP, and, in any event, performance should only be one factor of many in a CSP selection process. AFTER EDIT: and from the conclusion (which is worth reading): While Microsoft has secured the leadership position this year, it is quite possible that things could change again next year. In the two years this report has been published, there was a different leader each year. AFTER EDIT 2: From the second page of the article: Based on the findings of the report, Nasuni uses both Azure and AWS public cloud resources as part of the company's enterprise storage offering. If you don't want to fill in a form to read the report, there's an infographic: http://www.nasuni.com/blog/193-comparing_cloud_storage_providers_in http://www.nasuni.com/blog/193-comparing_cloud_storage_provi...
- eplanit 14y ago
  You beat me to it...I was primed to type "How much did Microsoft pay for this review?". That said, I cannot make any direct accusation. But, having been in this industry for >25 years, I did learn long ago to be very leery of such reviews when made by an industry "trade rag". I know from having worked for very large technology companies that, cynicism notwithstanding, reviews in trade rags are indeed often "bought".
  - Flenser 14y ago
    Indeed, from my quick scan of the report it looks like it was done in good faith. It could be that MS influenced Network World to publish it. Even if I knew they had I'd still be more favourably disposed towards Azure having read this report: Microsoft consistently performed better than the other CSPs in the tests, delivering the best Write/Read/Delete speeds across a variety of file sizes, the fastest response times and the fewest errors. Not only did Microsoft outperform the competition significantly during the raw performance tests, it was the only cloud storage platform to post zero errors during 100 million reads and writes. In those categories where Microsoft was not the top performer (uptime and scalability variance), it was a close second. In contrast, I'd much more wary of using Google Cloud Storage after reading this.
locusm 14y ago
There are a lot of things to like in Azure but performance isn't one of them. Disk IO is so dismal for VM's, we benchmarked against OrionVM and Zettagrid and the difference was about 10 to 1.
- locusm 14y ago
  Found it http://openbenchmarking.org/result/1206086-BY-1103157IV31 http://openbenchmarking.org/result/1206086-BY-1103157IV31
UnoriginalGuy 14y ago
Azure also seems cheaper than EC2 right now. Or at least in a couple of micro instance scenarios I just looked at they are... I might have to give Azure some time.
- OGinparadise 14y ago
  Azure also seems cheaper than EC2 right now. Probably by design, MS has a lot of leeway when it comes to spending thanks to Office, Windows and Enterprise. Amazon's profits are razor thin so MS might be trying to pressure them even more.
  - UnoriginalGuy 14y ago
    Maybe, but whatever keeps the market competitive is fine by me. Without Azure then Amazon has no reason to even re-evaluate their pricing.
tessa_82 14y ago
If you search hard enough you are bound to find a few vanity tests where your product will beat the competition, im sure Microsoft also have a whole series of searches for Bing that totally own Google. I dont buy into an isolated test funded by the company who will profit from performing well in it.
- nkerkin 14y ago
  Have you got a source for evidence that MS funded this test?
Flenser 14y ago
If you don't want to fill in a form to read the report, there's an infographic: http://www.nasuni.com/blog/193-comparing_cloud_storage_providers_in http://www.nasuni.com/blog/193-comparing_cloud_storage_provi...
jread 14y ago
This report is essentially a summary of network connectivity between 3 arbitrarily chosen external client VMs and 5 object storage platforms: AWS S3, Azure Blob storage, HP Object storage and Google developer storage. The results are highly dependent on network proximity between said clients and provider networks. With such a small client test population and no details provided about them (other than that they are on the east coast), I think the findings are inconclusive at best, and cetainly undeserving of these kinds of sensational headlines. Nansuni even acknowledges that the most optimal test conditions for object storage is use of clients within a provider's associated compute platform.

18 points by chermanowicz 14y ago

meaty 14y ago

That genuinely doesn't surprise me at all. I've used both extensively and Azure definitely has the edge with respect to performance (and management which is pretty neat). Also I've had severe reliability problems with S3 over the last couple of years. Azure support is actually better as well. I've found it very easy to get through to a human unlike when we had a major EC2 problem. However, I'd pick neither of them if I had to choose and would go with a smaller, old fashioned dedicated hosting company as you are rarely lost in the noise then if something goes bang. Prime example being Azure's reporting services which was dead for nearly an entire week.

rschmitty 14y ago
What kind of problems have you had with S3? I think we've all been affected by EBS in someway the last couple of years but I've not run into S3 issues. Could you elaborate?
- meaty 14y ago
  Connection reliability (we upload large files), the .net API is unreliable in the AWS SDK and lots of spurious issues. We've basically had to implement a retry wrapper around their API.

Flenser 14y ago

The first question I always have when I read headlines such as this is "Who paid for it?" I filled in the form for to download the report and it contains the following: Disclaimer The tests reported upon in this document are conducted by us using our own test tools under test conditions chosen by us. The test conditions were chosen by us to reasonably represent what our customers would experience using our Service with their representative environments and workloads. The tests have been designed by us to only look at the performance aspects of the CSPs that we believe are relevant to our customers – it is intentionally narrow in scope. Nasuni is not in the business of benchmarking CSPs, certifying test results or selling performance metrics. We have attempted to make sure the tests are fair and consistent within our selected parameters and have worked with several of the vendors to confirm our results. Our tests are not meant to indicate performance from each CSP under ideal conditions to the CSP, and, in any event, performance should only be one factor of many in a CSP selection process. AFTER EDIT: and from the conclusion (which is worth reading): While Microsoft has secured the leadership position this year, it is quite possible that things could change again next year. In the two years this report has been published, there was a different leader each year. AFTER EDIT 2: From the second page of the article: Based on the findings of the report, Nasuni uses both Azure and AWS public cloud resources as part of the company's enterprise storage offering. If you don't want to fill in a form to read the report, there's an infographic: http://www.nasuni.com/blog/193-comparing_cloud_storage_providers_in http://www.nasuni.com/blog/193-comparing_cloud_storage_provi...

eplanit 14y ago
You beat me to it...I was primed to type "How much did Microsoft pay for this review?". That said, I cannot make any direct accusation. But, having been in this industry for >25 years, I did learn long ago to be very leery of such reviews when made by an industry "trade rag". I know from having worked for very large technology companies that, cynicism notwithstanding, reviews in trade rags are indeed often "bought".
- Flenser 14y ago
  Indeed, from my quick scan of the report it looks like it was done in good faith. It could be that MS influenced Network World to publish it. Even if I knew they had I'd still be more favourably disposed towards Azure having read this report: Microsoft consistently performed better than the other CSPs in the tests, delivering the best Write/Read/Delete speeds across a variety of file sizes, the fastest response times and the fewest errors. Not only did Microsoft outperform the competition significantly during the raw performance tests, it was the only cloud storage platform to post zero errors during 100 million reads and writes. In those categories where Microsoft was not the top performer (uptime and scalability variance), it was a close second. In contrast, I'd much more wary of using Google Cloud Storage after reading this.

locusm 14y ago

There are a lot of things to like in Azure but performance isn't one of them. Disk IO is so dismal for VM's, we benchmarked against OrionVM and Zettagrid and the difference was about 10 to 1.

locusm 14y ago
Found it http://openbenchmarking.org/result/1206086-BY-1103157IV31 http://openbenchmarking.org/result/1206086-BY-1103157IV31

UnoriginalGuy 14y ago

Azure also seems cheaper than EC2 right now. Or at least in a couple of micro instance scenarios I just looked at they are... I might have to give Azure some time.

OGinparadise 14y ago
Azure also seems cheaper than EC2 right now. Probably by design, MS has a lot of leeway when it comes to spending thanks to Office, Windows and Enterprise. Amazon's profits are razor thin so MS might be trying to pressure them even more.
- UnoriginalGuy 14y ago
  Maybe, but whatever keeps the market competitive is fine by me. Without Azure then Amazon has no reason to even re-evaluate their pricing.

tessa_82 14y ago

If you search hard enough you are bound to find a few vanity tests where your product will beat the competition, im sure Microsoft also have a whole series of searches for Bing that totally own Google. I dont buy into an isolated test funded by the company who will profit from performing well in it.

nkerkin 14y ago
Have you got a source for evidence that MS funded this test?

Flenser 14y ago

If you don't want to fill in a form to read the report, there's an infographic: http://www.nasuni.com/blog/193-comparing_cloud_storage_providers_in http://www.nasuni.com/blog/193-comparing_cloud_storage_provi...

jread 14y ago

This report is essentially a summary of network connectivity between 3 arbitrarily chosen external client VMs and 5 object storage platforms: AWS S3, Azure Blob storage, HP Object storage and Google developer storage. The results are highly dependent on network proximity between said clients and provider networks. With such a small client test population and no details provided about them (other than that they are on the east coast), I think the findings are inconclusive at best, and cetainly undeserving of these kinds of sensational headlines. Nansuni even acknowledges that the most optimal test conditions for object storage is use of clients within a provider's associated compute platform.