5 ms·
I recently migrated some services from DigitalOcean to Fly and it was quite pleasant. My only issue is that routing to Fly from Southern California is awful. M
by sosodev 4y ago
I recently migrated some services from DigitalOcean to Fly and it was quite pleasant. My only issue is that routing to Fly from Southern California is awful.
My friend and I have different ISPs and yet we’re both routed to the east coast instead of the much closer Los Angeles edge server.
So our requests end up going to the east coast(edge), then back to the west coast (app), back to the east coast(edge), and then back to west coast (origin).
The latency adds up and makes requests to a nearby app almost as slow as to one hosted across an ocean.
I’m worried that other users will hit this and make hosting on Fly terrible for apps that require lower latency :/
- whalesalad 4y agoWhat ISP's are involved here?
- sosodev 4y agoCox and Spectrum. They route slightly different but both to the east coast. I linked the forum thread above.
- mrkurt 4y agoThis is uncommon. When did it happen? We had some cross country routing issues a few weeks ago. We can generally fix these if you don't mind sending us traveroutes.
- ryanisnan 4y agoThis is cool, I'd love to know more about how you generally fix issues like these?
- mrkurt 4y agoI usually just go complain to network folks and they figure it out. I'm good at complaining! Routing issues like this are typically the result of some weird network provider politics. The fix is typically ti change how we announce IPs to force a route to do something we want. It's a dark art.
- ryanisnan 4y agoAwesome, thanks for sharing. I am also good at complaining! We might make good friends.
- sosodev 4y agoHere’s the forum thread I made about the issue. https://community.fly.io/t/how-is-edge-routing-determined/9608/2 https://community.fly.io/t/how-is-edge-routing-determined/96... Traces and ISP involved are in there. Thanks for looking into it! Edit: I just double checked and it's still the same route to the east coast as posted in the thread
- mrkurt 4y agoAh, thanks. We'll chase it down. We got a little behind over the holidays.
- sosodev 4y agoI really do appreciate it. Is there any way Fly could detect this? I know geo IP isn’t perfect but it seems like it could help you track this stuff by allowing you to compare edge location to the users location.
- tptacek 4y agoThere probably is a good way to do this. Do you have thoughts on things we should do here? We have a code footprint in every region (and also in like 6 off-platform regions on other providers) for doing this kind of monitoring, and we're probably not putting it to the most effective use we can. Ideas would be most welcome.
- sosodev 4y agoThis idea might be a bit too naive but it seems like it could help 1) Record the IP address and edge of connecting clients 2) Use an IP API to determine the rough location and ISP of clients based on their IP 3) Spatially query the edge closest to each client and graph/log serious mismatches (>x miles off) 4) Cast the spells required to route offending ISP better 5) See if the data improves :) For example, the IAD edge would record that I connected from San Diego via Cox Communications. My closest edge geographically is LAX so that's ~2500 miles of excess travel!