4 ms·

AWS Olympics: Speed-Testing EC2 and S3 Across Regions

55 points by chookrl 14y ago

tawgx 14y ago
Thanks for putting this up. I'd be very intrested to know if anyone has ever done a similar test with Azure?
- chookrl 14y ago
  We haven't tested Azure Cloud Storage. However, we did test other cloud computing hosts such as GoGrid and Rackspace. Their upload speeds were almost the same and sometimes even better than EC2 when uploading to AWS S3. Perhaps we will test Azure in a follow-up post.
eran_rl75 14y ago
Interesting!! We've tried conducting similar tests where I work a few months ago. We had to deploy a few servers and tried to figure out the optimal spread.
dbarlett 14y ago
For anyone interested in optimizing S3 performance, I recommend Jim Sorenson's "Building Scalable Applications on Amazon S3" from re:Invent. http://www.slideshare.net/AmazonWebServices/building-scalable-applications-on-amazon-s3-stg303 http://www.slideshare.net/AmazonWebServices/building-scalabl... http://www.youtube.com/watch?v=YYnVRYbUR6A http://www.youtube.com/watch?v=YYnVRYbUR6A
- nachteilig 14y ago
  His comments on hashing for S3 buckets were pretty interesting.
  - res0nat0r 14y ago
    Here's a very informative blog post by Doug Grismore that talks more about this if you are interested: http://aws.typepad.com/aws/2012/03/amazon-s3-performance-tips-tricks-seattle-hiring-event.html http://aws.typepad.com/aws/2012/03/amazon-s3-performance-tip...
blahpro 14y ago
Interesting. It might also be useful to see latencies between regions (or even availability zones).
cpa 14y ago
Any idea why the results are not symmetrical? Upload from Virginia to California is twice as fast as from California to Viriginia (and it's even worse with US <-> Japan). Are there strategical interests to not have a symmetric link?
- Xylakant 14y ago
  Maybe the link is symmetric, but the traffic flow is not - so in one direction the intertubes are more clogged.
- datasage 14y ago
  Actually, the results are reverse from what you said (y-axis is the origin). It likely has to do with traffic congestion coming out of each datacenter. Virginia has by far the heaviest use out of all the datacenters.
datasage 14y ago
Australia is has low bandwidth to the rest of the world largely due its geography and capacity/path of the undersea cables connecting the country. Geographically its relatively close to Singapore (compared to other data centers anyway), all of the cables from the eastern side of the country either go up to Japan via Guam or go to the US via Hawaii. Its not uncommon to see a route from Sydney to Singapore go through the US, and/or Japan before reaching Singapore.
toki5 14y ago
You know, I know it's 2013 and all, and things like this shouldn't amaze me, but the fact that you can get data from Australia to Brazil in _minutes_ is still kind of astounding to me.
oscarwao 14y ago
Keep in mind that the S3 Virgnia location (known as US Standard Regionin AWS terms) includes endpoints in Virginia and in the Pacific Northwest. Hence that may be why the Virginia to Virginia numbers were high compared to the other regions. "The US Standard Region automatically routes requests to facilities in Northern Virginia or the Pacific Northwest using network maps." http://aws.amazon.com/s3/ http://aws.amazon.com/s3/
- jedberg 14y ago
  If that's the cause of slowness then their network maps are broken.
- donavanm 14y ago
  The us-east and SEA end points for US Standard buckets are directly addressable as well. Consistency will be tighter if you PUT and GET to the same endpoint. Edit: that's be s3-external-1.amazonaws.com for us-east and s3-external-2.amazonaws.com for the PNW. See https://forums.aws.amazon.com/message.jspa?messageID=185820 https://forums.aws.amazon.com/message.jspa?messageID=185820

55 points by chookrl 14y ago

tawgx 14y ago

Thanks for putting this up. I'd be very intrested to know if anyone has ever done a similar test with Azure?

chookrl 14y ago
We haven't tested Azure Cloud Storage. However, we did test other cloud computing hosts such as GoGrid and Rackspace. Their upload speeds were almost the same and sometimes even better than EC2 when uploading to AWS S3. Perhaps we will test Azure in a follow-up post.

eran_rl75 14y ago

Interesting!! We've tried conducting similar tests where I work a few months ago. We had to deploy a few servers and tried to figure out the optimal spread.

dbarlett 14y ago

For anyone interested in optimizing S3 performance, I recommend Jim Sorenson's "Building Scalable Applications on Amazon S3" from re:Invent. http://www.slideshare.net/AmazonWebServices/building-scalable-applications-on-amazon-s3-stg303 http://www.slideshare.net/AmazonWebServices/building-scalabl... http://www.youtube.com/watch?v=YYnVRYbUR6A http://www.youtube.com/watch?v=YYnVRYbUR6A

nachteilig 14y ago
His comments on hashing for S3 buckets were pretty interesting.
- res0nat0r 14y ago
  Here's a very informative blog post by Doug Grismore that talks more about this if you are interested: http://aws.typepad.com/aws/2012/03/amazon-s3-performance-tips-tricks-seattle-hiring-event.html http://aws.typepad.com/aws/2012/03/amazon-s3-performance-tip...

blahpro 14y ago

Interesting. It might also be useful to see latencies between regions (or even availability zones).

cpa 14y ago

Any idea why the results are not symmetrical? Upload from Virginia to California is twice as fast as from California to Viriginia (and it's even worse with US <-> Japan). Are there strategical interests to not have a symmetric link?

Xylakant 14y ago
Maybe the link is symmetric, but the traffic flow is not - so in one direction the intertubes are more clogged.
datasage 14y ago
Actually, the results are reverse from what you said (y-axis is the origin). It likely has to do with traffic congestion coming out of each datacenter. Virginia has by far the heaviest use out of all the datacenters.

datasage 14y ago

Australia is has low bandwidth to the rest of the world largely due its geography and capacity/path of the undersea cables connecting the country. Geographically its relatively close to Singapore (compared to other data centers anyway), all of the cables from the eastern side of the country either go up to Japan via Guam or go to the US via Hawaii. Its not uncommon to see a route from Sydney to Singapore go through the US, and/or Japan before reaching Singapore.

toki5 14y ago

You know, I know it's 2013 and all, and things like this shouldn't amaze me, but the fact that you can get data from Australia to Brazil in _minutes_ is still kind of astounding to me.

oscarwao 14y ago

Keep in mind that the S3 Virgnia location (known as US Standard Regionin AWS terms) includes endpoints in Virginia and in the Pacific Northwest. Hence that may be why the Virginia to Virginia numbers were high compared to the other regions. "The US Standard Region automatically routes requests to facilities in Northern Virginia or the Pacific Northwest using network maps." http://aws.amazon.com/s3/ http://aws.amazon.com/s3/

jedberg 14y ago
If that's the cause of slowness then their network maps are broken.
donavanm 14y ago
The us-east and SEA end points for US Standard buckets are directly addressable as well. Consistency will be tighter if you PUT and GET to the same endpoint. Edit: that's be s3-external-1.amazonaws.com for us-east and s3-external-2.amazonaws.com for the PNW. See https://forums.aws.amazon.com/message.jspa?messageID=185820 https://forums.aws.amazon.com/message.jspa?messageID=185820