3 ms·

Does 2x100G mean 100G each way?

by ummonk 5y ago

martinald 5y ago
No, 200 each way (2 100gbit links). Not really sure why this is a big deal - IX bandwidth is very very cheap. It looks like seattleIX charges $7500 as a one off payment to add a 100gbit link: https://www.seattleix.net/pr20191218 https://www.seattleix.net/pr20191218).
- toast0 5y ago
  Mostly, network connections aren't so public, so it's interesting in general. Also, upgrading their IX capacity may be a sign that their product is maturing or has significant use. As an upstart ISP, Starlink needs to seek out the connections to CDNs that their customers demand, and an IX connection is a good place to get them at substantially less cost than transit.
  - ccakes 5y ago
    Transit is very cheap these days if you're buying 100G ports.. the economics of whether to peer or not aren't as clear cut as they once were.
- sbierwagen 5y ago
  >IX bandwidth is very very cheap I would describe it as "capacity", not bandwidth. That fee is just to physically plug into the router. You have to make separate agreements with the other organizations also plugged into that router for peering or paid transit.
toast0 5y ago
No, at this level, connections are symetric. The next question is probably why is it reported as 2x2x100G? That's most likely two separate connections which are each an aggregated connection of two 100G connections. This is likely to two separate routers on the Starlink.
- jpgvm 5y ago
  More importantly it's probably to 2 redundant routers on SIX side also so that both can handle HW failure. On the SIX side the ports are probably already in a redundant chassis but having 2 ports probably still scores you connections to two fully independent chassis in different failure zones.
  - kloch 5y ago
    The last time I got a tour of the SIX, there were switches on two different floors in the Westin building. This is more for capacity than redundancy. The original location is literally a janitor's closet in a hallway and it's so tight that exhaust fans are built into the door. The second location was in proper four-post racks in a small room. It is possible they have ports on both switches for redundancy but more likely both are on the newer location. The SIX is also built into other buildings and they could connect to more than one, but there is typically only 200G backhaul to the other sites. https://www.seattleix.net/topology https://www.seattleix.net/topology 2x100G to an IX are usually connected as separate ports, each with their own assigned IP ont he IX subnet. BGP multipath can enable load balancing across both ports. Otherwise you might have some peers configured on one port and others on the second. These days I like to have all peers on both ports and use BGP multipath. If both ports are connected to a single switch on the IX side and the same router on the customer side then LACP can be used to combine them into a single 200G port with one IP. This is a less common configuration unless you are doing more than two ports (4x100G as 2x 2x100G lags for example). There is another consideration: what happens if there is an icmp or arp flood on the IX? It's not supposed to happen but sometimes it does, even on the most major and well-run exchanges. In that case you might want to have all of your IX ports on one of your routers, keeping the other one "clean", just in case your control plane policing doesn't mitigate it. Different networks place different priority on IX ports. In most cases it's ok if it's down for a short time (traffic will just shift to paid transit) so redundancy is not essential. In other cases it may be too much traffic to reroute so you care more about IX reliability and redundancy.
    - jpgvm 5y ago
      Just want to say really great summary. I know these things but I bet many folks reading this glean an insight or two. Been a lot time since I have done all this stuff though. I sort of miss it sometimes (though not the pager that comes with the territory).
martyvis 5y ago
Most network links imply duplex. From this (and the description detail in the OP), I would say they have added 2 links (to the 2 they had previously), each link will have 100Gbps up and down through the IX.

Does 2x100G mean 100G each way?

by ummonk 5y ago

Does 2x100G mean 100G each way?

No, 200 each way (2 100gbit links). Not really sure why this is a big deal - IX bandwidth is very very cheap. It looks like seattleIX charges $7500 as a one off payment to add a 100gbit link: https://www.seattleix.net/pr20191218 https://www.seattleix.net/pr20191218).

toast0 5y ago
Mostly, network connections aren't so public, so it's interesting in general. Also, upgrading their IX capacity may be a sign that their product is maturing or has significant use. As an upstart ISP, Starlink needs to seek out the connections to CDNs that their customers demand, and an IX connection is a good place to get them at substantially less cost than transit.
- ccakes 5y ago
  Transit is very cheap these days if you're buying 100G ports.. the economics of whether to peer or not aren't as clear cut as they once were.
sbierwagen 5y ago
>IX bandwidth is very very cheap I would describe it as "capacity", not bandwidth. That fee is just to physically plug into the router. You have to make separate agreements with the other organizations also plugged into that router for peering or paid transit.

toast0 5y ago

No, at this level, connections are symetric. The next question is probably why is it reported as 2x2x100G? That's most likely two separate connections which are each an aggregated connection of two 100G connections. This is likely to two separate routers on the Starlink.

jpgvm 5y ago
More importantly it's probably to 2 redundant routers on SIX side also so that both can handle HW failure. On the SIX side the ports are probably already in a redundant chassis but having 2 ports probably still scores you connections to two fully independent chassis in different failure zones.
- kloch 5y ago
  The last time I got a tour of the SIX, there were switches on two different floors in the Westin building. This is more for capacity than redundancy. The original location is literally a janitor's closet in a hallway and it's so tight that exhaust fans are built into the door. The second location was in proper four-post racks in a small room. It is possible they have ports on both switches for redundancy but more likely both are on the newer location. The SIX is also built into other buildings and they could connect to more than one, but there is typically only 200G backhaul to the other sites. https://www.seattleix.net/topology https://www.seattleix.net/topology 2x100G to an IX are usually connected as separate ports, each with their own assigned IP ont he IX subnet. BGP multipath can enable load balancing across both ports. Otherwise you might have some peers configured on one port and others on the second. These days I like to have all peers on both ports and use BGP multipath. If both ports are connected to a single switch on the IX side and the same router on the customer side then LACP can be used to combine them into a single 200G port with one IP. This is a less common configuration unless you are doing more than two ports (4x100G as 2x 2x100G lags for example). There is another consideration: what happens if there is an icmp or arp flood on the IX? It's not supposed to happen but sometimes it does, even on the most major and well-run exchanges. In that case you might want to have all of your IX ports on one of your routers, keeping the other one "clean", just in case your control plane policing doesn't mitigate it. Different networks place different priority on IX ports. In most cases it's ok if it's down for a short time (traffic will just shift to paid transit) so redundancy is not essential. In other cases it may be too much traffic to reroute so you care more about IX reliability and redundancy.
  - jpgvm 5y ago
    Just want to say really great summary. I know these things but I bet many folks reading this glean an insight or two. Been a lot time since I have done all this stuff though. I sort of miss it sometimes (though not the pager that comes with the territory).

martyvis 5y ago

Most network links imply duplex. From this (and the description detail in the OP), I would say they have added 2 links (to the 2 they had previously), each link will have 100Gbps up and down through the IX.