3 ms·

Starlink says on their FAQ page[1] for the roaming model that if you roam too far, your connection will be briefly interrupted while they assign you a new IP ad

Starlink says on their FAQ page[1] for the roaming model that if you roam too far, your connection will be briefly interrupted while they assign you a new IP address. I think that's because they want every user to have a reasonable experience with all the websites out there that do geoIP lookups. It also gives them flexibility in the future to not act as a worldwide internet backbone for their own users traffic by advertising each set of local IP's only at local POP's. [1]: https://starlink-enterprise-guide.readme.io/docs/dhcp-configuration#:~:text=If%20you%20have%20a%20mobile%20Starlink%20the%20IP%20address%20may%20change%20if%20you%20move%20locations.%20These%20changes%20will%20cause%20a%20brief%20DHCP%20handoff%20outage https://starlink-enterprise-guide.readme.io/docs/dhcp-config....

lxgr 3y ago
> It also gives them flexibility in the future to not act as a worldwide internet backbone for their own users traffic by advertising each set of local IP's only at local POP's. They'd need multiple ASNs for each of these network islands then, though – is their network actually set up like that? > if you roam too far, your connection will be briefly interrupted while they assign you a new IP address. That's an interesting detail: It seems like they might not actually have cross base station roaming/connectivity, unlike e.g. mobile ISPs in large countries (where you can keep the same IP and maintain open connections even as you're driving across the country). I wonder if Starlink will eventually build out a terrestrial backbone to complement their satellite network for these two scenarios (i.e. very mobile base stations, such as those on airplanes, and terminating their own traffic globally as opposed to locally).
- inemesitaffia 3y ago
  Starlink has one ASN.
- ccakes 3y ago
  > They'd need multiple ASNs for each of these network islands then, though No they wouldn’t. They could connect to local transit/IXs in each region and just advertise the regional networks.
  - lxgr 3y ago
    Oh, I thought it wasn't possible to announce multiple isolated networks on the same ASN without interconnecting them, since ASNs are the routing granularity of BGP?
    - ericpauley 3y ago
      One counterexample to this would be Amazon, who have IPs globally under one ASN. While they do have a global backhaul, it seems unlikely they accept traffic for any IP anywhere and route it all on the backhaul.
      - dilyevsky 3y ago
        > accept traffic for any IP anywhere and route it all on the backhaul. This is actually what they do when you enable Global Accelerator which is a paid feature. In GCP they have similar concept called “Premium Tier”
        ericpauley 3y ago
        True! In my post I meant for all traffic.
    - zajio1am 3y ago
      AS is basic granularity in BGP in the sense that one ASN should not be multiple times in AS PATH, and it is generally assumed that ASs are internally connected (although for non-transit networks, this could be workarounded in several ways) But basic routing entity in BGP is still prefix/route, and different routes of one AS can be announced in different ways.
    - Hikikomori 3y ago
      For something remote it just looks like two different prefixes with a different path to reach them. It's even a commonly used technique for multi homing so you can control which peer ingress traffic comes in on for a specific prefix.
  - 3y ago
- jrockway 3y ago
  I don't think you need multiple ASNs for this. The "server" end handles this; if your packet bound for 8.8.8.8 leaves Starlink's network in Tulsa, then you get 8.8.8.8's server in Tulsa (because that is the route that Google advertised to Starlink at that PoP), then the 8.8.8.8 server in Tulsa talks to your own DNS server in Houston, and you only serve the IPs for your Houston datacenter in that reply as a matter of policy. (You might not run the DNS server, of course, but that's what, say, Cloudflare would do for you.) Then an Internet-wide route controls further traffic; if Starlink decides to make that customer's traffic egress in New York, then that packet is stuck going cross-country. It doesn't have to be that way; you could host your personal website on anycast if you wanted to, but it's a lot of work. You don't need an ASN per datacenter, though, you just need to control your route advertisements on a per-datacenter basis. GeoIP is a further complication not really related to the Internet. The DNS server in Houston can serve replies for packets coming from anywhere in the country; instead of looking at what network interface they arrived on, it looks in a database, decides 1.2.3.4 is in Oklahoma, and provides IP addresses for the Houston datacenter because of that. For Starlink to provide the best path in that case, it needs to understand what that database looks like; it's likely they populate it themselves in a protocol outside from any Internet machinery. (The most naive databases just look at the ASN of the source address, look at where the address of the ISP is, and decides that's the location of the IP address. But that is too naive to work, so nobody does that exactly.) I would imagine that most of the CDNs / edge compute providers use some combination of these two data points to route traffic. I have never set anything like this up, so I'm not sure what the state of the art is. But, there are many tools that you can use, and none of them require having an extra ASN. When I worked at an ISP we did some somewhat-sketchy management around this. The databases often have a flag for "is this address a cloud provider or a residential ISP subscriber?" We would see which of our IP ranges were considered residential (we had 0 residential customers), and then use them for IP transit subscribers that wanted to sell services like scraping Amazon or whatever. Amazon would check their database, say "oh this is just a virus-infected cable modem", and allow the traffic. If the reputation changed, we could just give the address to a transit customer that didn't want to do that kind of stuff. Just going to say, I played no part in this business decision; it existed when I arrived and existed when I left. Gotta pay back the investors somehow, I guess! (At my next job I was on the other side of this; trying to combat free trial abuse. The databases were largely useless; what happens is that people at high-reputation ISPs get their servers hacked and the attacks end up launched from there. The big cloud providers were like 100% abuse traffic, probably because of free trials of CI systems and things like that. Ultimately, most traffic going through a US-based transit provider got "please contact sales to do a free trial". Actual residential ISPs like Comcast were totally fine; thanks Windows Update! Since it was the pandemic, everyone was working from home, and thus the business ISPs being gated behind "talk to sales" were not a big problem. Oh and BTW, I am certain that most of the abuse was from HN readers; it started the day after we mentioned our product here. That cat and mouse game was some of the most fun I've had at work!)
- extraduder_ire 3y ago
  > I wonder if Starlink will eventually build out a terrestrial backbone to complement their satellite network for these two scenarios (i.e. very mobile base stations, such as those on airplanes, and terminating their own traffic globally as opposed to locally). I think their stated plan is to use laser-links to satellites in higher orbits to do that. Don't know if they'll be bandwidth limited, but there's better latency doing it this way. (light goes faster in the vacuum of space, vs going down an optical fibre)

lxgr 3y ago

> It also gives them flexibility in the future to not act as a worldwide internet backbone for their own users traffic by advertising each set of local IP's only at local POP's. They'd need multiple ASNs for each of these network islands then, though – is their network actually set up like that? > if you roam too far, your connection will be briefly interrupted while they assign you a new IP address. That's an interesting detail: It seems like they might not actually have cross base station roaming/connectivity, unlike e.g. mobile ISPs in large countries (where you can keep the same IP and maintain open connections even as you're driving across the country). I wonder if Starlink will eventually build out a terrestrial backbone to complement their satellite network for these two scenarios (i.e. very mobile base stations, such as those on airplanes, and terminating their own traffic globally as opposed to locally).

inemesitaffia 3y ago
Starlink has one ASN.
ccakes 3y ago
> They'd need multiple ASNs for each of these network islands then, though No they wouldn’t. They could connect to local transit/IXs in each region and just advertise the regional networks.
- lxgr 3y ago
  Oh, I thought it wasn't possible to announce multiple isolated networks on the same ASN without interconnecting them, since ASNs are the routing granularity of BGP?
  - ericpauley 3y ago
    One counterexample to this would be Amazon, who have IPs globally under one ASN. While they do have a global backhaul, it seems unlikely they accept traffic for any IP anywhere and route it all on the backhaul.
    - dilyevsky 3y ago
      > accept traffic for any IP anywhere and route it all on the backhaul. This is actually what they do when you enable Global Accelerator which is a paid feature. In GCP they have similar concept called “Premium Tier”
      - ericpauley 3y ago
        True! In my post I meant for all traffic.
  - zajio1am 3y ago
    AS is basic granularity in BGP in the sense that one ASN should not be multiple times in AS PATH, and it is generally assumed that ASs are internally connected (although for non-transit networks, this could be workarounded in several ways) But basic routing entity in BGP is still prefix/route, and different routes of one AS can be announced in different ways.
  - Hikikomori 3y ago
    For something remote it just looks like two different prefixes with a different path to reach them. It's even a commonly used technique for multi homing so you can control which peer ingress traffic comes in on for a specific prefix.
- 3y ago
jrockway 3y ago
I don't think you need multiple ASNs for this. The "server" end handles this; if your packet bound for 8.8.8.8 leaves Starlink's network in Tulsa, then you get 8.8.8.8's server in Tulsa (because that is the route that Google advertised to Starlink at that PoP), then the 8.8.8.8 server in Tulsa talks to your own DNS server in Houston, and you only serve the IPs for your Houston datacenter in that reply as a matter of policy. (You might not run the DNS server, of course, but that's what, say, Cloudflare would do for you.) Then an Internet-wide route controls further traffic; if Starlink decides to make that customer's traffic egress in New York, then that packet is stuck going cross-country. It doesn't have to be that way; you could host your personal website on anycast if you wanted to, but it's a lot of work. You don't need an ASN per datacenter, though, you just need to control your route advertisements on a per-datacenter basis. GeoIP is a further complication not really related to the Internet. The DNS server in Houston can serve replies for packets coming from anywhere in the country; instead of looking at what network interface they arrived on, it looks in a database, decides 1.2.3.4 is in Oklahoma, and provides IP addresses for the Houston datacenter because of that. For Starlink to provide the best path in that case, it needs to understand what that database looks like; it's likely they populate it themselves in a protocol outside from any Internet machinery. (The most naive databases just look at the ASN of the source address, look at where the address of the ISP is, and decides that's the location of the IP address. But that is too naive to work, so nobody does that exactly.) I would imagine that most of the CDNs / edge compute providers use some combination of these two data points to route traffic. I have never set anything like this up, so I'm not sure what the state of the art is. But, there are many tools that you can use, and none of them require having an extra ASN. When I worked at an ISP we did some somewhat-sketchy management around this. The databases often have a flag for "is this address a cloud provider or a residential ISP subscriber?" We would see which of our IP ranges were considered residential (we had 0 residential customers), and then use them for IP transit subscribers that wanted to sell services like scraping Amazon or whatever. Amazon would check their database, say "oh this is just a virus-infected cable modem", and allow the traffic. If the reputation changed, we could just give the address to a transit customer that didn't want to do that kind of stuff. Just going to say, I played no part in this business decision; it existed when I arrived and existed when I left. Gotta pay back the investors somehow, I guess! (At my next job I was on the other side of this; trying to combat free trial abuse. The databases were largely useless; what happens is that people at high-reputation ISPs get their servers hacked and the attacks end up launched from there. The big cloud providers were like 100% abuse traffic, probably because of free trials of CI systems and things like that. Ultimately, most traffic going through a US-based transit provider got "please contact sales to do a free trial". Actual residential ISPs like Comcast were totally fine; thanks Windows Update! Since it was the pandemic, everyone was working from home, and thus the business ISPs being gated behind "talk to sales" were not a big problem. Oh and BTW, I am certain that most of the abuse was from HN readers; it started the day after we mentioned our product here. That cat and mouse game was some of the most fun I've had at work!)
extraduder_ire 3y ago
> I wonder if Starlink will eventually build out a terrestrial backbone to complement their satellite network for these two scenarios (i.e. very mobile base stations, such as those on airplanes, and terminating their own traffic globally as opposed to locally). I think their stated plan is to use laser-links to satellites in higher orbits to do that. Don't know if they'll be bandwidth limited, but there's better latency doing it this way. (light goes faster in the vacuum of space, vs going down an optical fibre)