4 ms·
I used to read ThousandEyes's annual free reports on networking capabilities and measurements across the Big 3 cloud providers, which were quite comprehensive i
by ignoramous 6y ago
I used to read ThousandEyes's annual free reports on networking capabilities and measurements across the Big 3 cloud providers, which were quite comprehensive in terms of covering various scenarios [0]. I don't think they publish those anymore.
With cloudping.co, it isn't clear from the website, but it seems like the inter-region latency they measure is over the public Internet. The Big 3 run their own backbone across all their DCs around the globe, and so I reckon, those numbers would look vastly different if traffic was instead relayed through those uncongested backbones.
With AWS, the cheapest way I know (in terms of development time and cost) to accomplish inter-region over their backbone is via Global Load Accelerator. The clients connect to GLA at the nearest anycast IP location advertised across 150+ AWS PoPs. You can then play with endpoint-groups, connection-affinities, source/port tuples to control routing traffic to different backends in various regions.
We used this technique to prototype a VPN with exit nodes in multiple countries but entry nodes closest to clients at every AWS PoP. It worked quite nicely for a toy: https://news.ycombinator.com/item?id=21071593 https://news.ycombinator.com/item?id=21071593
[0] https://www.thousandeyes.com/blog/top-takeaways-cloud-performance-benchmark https://www.thousandeyes.com/blog/top-takeaways-cloud-perfor... (2019)
- virtuallynathan 6y ago> With cloudping.co, it isn't clear from the website, but it seems like the inter-region latency they measure is over the public Internet If they are running from AWS to AWS, it's going over the AWS backbone.
- toredash 6y agoIs that true? I thought you had to do VPC peering for regions to use those backbones
- ignoramous 6y agoYeah, you need any among VPC Peering, Private Link, Transit Gateway for inter-region (global) backbone connections.
- kitteh 6y agoInter region traffic always goes over the backbone (this includes EIP to EIP). This also includes going from EC2 to any service like S3 in another region. Except China. China to rest of world is not via backbone.
- toredash 6y agoI don't see this in the docs
- geoduck14 6y agoChina is super special. Stay out of that region unless you have a special China reason to go in there.
- kitteh 6y agoIt should be. Also this was covered in a reinvent presentation about the aws network.
- ignoramous 6y agoI doubt unless you're using VPC peering, Transit Gateway, or Private Link that it would be the case that user-generated traffic between regions (for ex, between EC2 instances in Dublin and Sydney) is automatically routed through their backbone. Can you point to the re:Invent presentation? Genuinely curious.
- virtuallynathan 6y agoIt really is, for all AWS-AWS traffic barring Beijing and Ningxia. src: I worked in AWS networking for 2.5 years here's DUB->SYD: HOST: amazon02.ring.nlnog.net Loss% Snt Last Avg Best Wrst StDev 1. AS16509 ec2-3-248-240-73.eu 0.0% 10 7.1 22.8 1.0 125.7 38.1 2. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 3. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 4. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 5. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 6. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 7. AS??? 100.65.15.3 0.0% 10 0.8 1.1 0.2 6.7 2.0 8. AS??? 100.95.19.145 0.0% 10 0.3 1.4 0.2 5.5 1.9 9. AS??? 100.100.4.12 0.0% 10 0.3 1.1 0.3 7.0 2.1 10. AS??? 150.222.242.237 0.0% 10 254.0 254.0 253.9 254.4 0.2 11. AS??? 52.95.36.164 0.0% 10 257.2 255.7 254.0 257.3 1.3 12. AS??? 150.222.112.139 0.0% 10 253.4 253.9 253.4 255.1 0.5 13. AS??? 150.222.112.142 0.0% 10 259.6 257.8 255.5 266.8 3.5 14. AS??? 52.95.36.143 0.0% 10 255.3 256.1 255.2 261.4 1.9 15. AS??? 52.95.38.17 0.0% 10 255.0 255.6 254.9 259.9 1.6 16. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 17. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 18. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 19. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 20. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 21. AS??? 100.65.16.65 0.0% 10 256.7 279.8 256.7 334.4 26.1 22. AS16509 amazon08.ring.nlnog 0.0% 10 254.4 254.4 254.3 254.4 0.0