3 ms·

Obviously the biggest contribution to latency is distance. But there's also some close-ish regions with poor latency because there's not fiber running directly

Obviously the biggest contribution to latency is distance. But there's also some close-ish regions with poor latency because there's not fiber running directly between them (for example, over the poles) Are there an examples of regions which dramatically violate the triangle equality? (That is, where the A--C latency is much worse than the best A--B + B--C latencies)? Just as a curiosity, could you use that idea to "infer" which data-centers are most likely directly connected by fiber, and show only the likely fiber connections?

llm_trw 2y ago
You can just look at the map of fiber optic cables around the world: https://www.submarinecablemap.com/ https://www.submarinecablemap.com/ It's highly unlikely there are any non-disclosed undersea ones since they cost rather a lot to lay down.
- defrost 2y ago
  > It's highly unlikely there are any .. Going back decades when a billion US was real money the original NSA (No Such Agency) that essentially no one had ever heard of, including most of the US houses and much of the defence committees that had clearance but not that clearance, had a 4 Billion+ budget for "off-book" satellites. Black cables are a damn sight cheaper than black satellites.
- curtisf 2y ago
  Yes, although I think it would also clean up the visualization, since you wouldn't have nearly so many lines connecting data centers which actually aren't connected; and it would therefore also be explanatory
kqr 2y ago
> Are there an examples of regions which dramatically violate the triangle equality? (That is, where the A--C latency is much worse than the best A--B + B--C latencies)? I don't think this would happen at a significant scale, due to how routing works. If taking the "detour" through B is how the ICMP packets get there cheapest, that's the path they will go. If anything, we could look at where A–C is nearly equal to A–B + B–C and find where such a thing has happened. I suppose it could happen for reasons other than lack of fiber: financially better peering agreements, etc?
maxmouchet 2y ago
I can't speak for AWS specifically, but in my PhD thesis [1] I found a bunch of such examples by using RIPE Atlas probes. Essentially looking for pairs of probes where the RTT between probe A and probe C is larger than probe A-B + B-C. Now there are some issues with this methodology (all common issues with ICMP/RTT measurements + traffic was not really routed through the "relay" probe), but such pairs do exist. [1] https://theses.hal.science/tel-03666771/document https://theses.hal.science/tel-03666771/document (see page 84 for an example; if you can read French :-))
Thaxll 2y ago
There are some regions that have notoriously bad networking with higher packet loss, for example South America and South Asia are pretty bad overall.
bfelbo 2y ago
There’s very few and poor cables in the areas between Russia/Mongolia/India. AFAIK, the latency from Mumbai to southern Russia (not that far in distance) is surprisingly high. Much higher than from e.g. Frankfurt to Moscow. Don’t know if it’s enough to violate the triangle equality between Frankfurt-Moscow-Mumbai.

llm_trw 2y ago

You can just look at the map of fiber optic cables around the world: https://www.submarinecablemap.com/ https://www.submarinecablemap.com/ It's highly unlikely there are any non-disclosed undersea ones since they cost rather a lot to lay down.

defrost 2y ago
> It's highly unlikely there are any .. Going back decades when a billion US was real money the original NSA (No Such Agency) that essentially no one had ever heard of, including most of the US houses and much of the defence committees that had clearance but not that clearance, had a 4 Billion+ budget for "off-book" satellites. Black cables are a damn sight cheaper than black satellites.
curtisf 2y ago
Yes, although I think it would also clean up the visualization, since you wouldn't have nearly so many lines connecting data centers which actually aren't connected; and it would therefore also be explanatory

kqr 2y ago

> Are there an examples of regions which dramatically violate the triangle equality? (That is, where the A--C latency is much worse than the best A--B + B--C latencies)? I don't think this would happen at a significant scale, due to how routing works. If taking the "detour" through B is how the ICMP packets get there cheapest, that's the path they will go. If anything, we could look at where A–C is nearly equal to A–B + B–C and find where such a thing has happened. I suppose it could happen for reasons other than lack of fiber: financially better peering agreements, etc?

maxmouchet 2y ago

I can't speak for AWS specifically, but in my PhD thesis [1] I found a bunch of such examples by using RIPE Atlas probes. Essentially looking for pairs of probes where the RTT between probe A and probe C is larger than probe A-B + B-C. Now there are some issues with this methodology (all common issues with ICMP/RTT measurements + traffic was not really routed through the "relay" probe), but such pairs do exist. [1] https://theses.hal.science/tel-03666771/document https://theses.hal.science/tel-03666771/document (see page 84 for an example; if you can read French :-))

Thaxll 2y ago

There are some regions that have notoriously bad networking with higher packet loss, for example South America and South Asia are pretty bad overall.

bfelbo 2y ago

There’s very few and poor cables in the areas between Russia/Mongolia/India. AFAIK, the latency from Mumbai to southern Russia (not that far in distance) is surprisingly high. Much higher than from e.g. Frankfurt to Moscow. Don’t know if it’s enough to violate the triangle equality between Frankfurt-Moscow-Mumbai.