5 ms·

Public internet exchanges are huge in Europe, you can basically reach every network in Europe by connecting to a few large exchanges for a fraction of what you'

by cbg0 6y ago

Public internet exchanges are huge in Europe, you can basically reach every network in Europe by connecting to a few large exchanges for a fraction of what you'd pay for tier 1 transit (granted, you still probably need it to reach some eyeballs). I wish we'd see the same concept pop up in the US, as these exchanges are definitely not as popular there. https://www.de-cix.net/en/locations/germany/frankfurt/statistics https://www.de-cix.net/en/locations/germany/frankfurt/statis... https://www.ams-ix.net/ams/documentation/total-stats https://www.ams-ix.net/ams/documentation/total-stats https://portal.linx.net/ https://portal.linx.net/

sneak 6y ago
A big part of it is that simply connecting your own network to these exchanges is a lot more expensive in the US due to increased land size and corrupt legislation. Due to the intense regulations (which are bought and paid for at the legislative level, and mostly exist to protect the large incumbents like Verizon, AT&T, Comcast, CenturyLink, et al from any new competitors) you end up having to buy your "last mile" loop (which, due to the US's size and general car-centric design, is much more than just a mile in most cases) that connects your network (in your office, or data center, or whatever, over to the exchange point) from a small number of monopolists. They're thousands, or tens of thousands of dollars per month. "We don't care. We don't have to. We're the phone company."
gbil 6y ago
Internet Exchanges doesn't mean that a peering agreement is also in place automatically. For example: https://www.ams-ix.net/ams/documentation/general-terms-and-conditions https://www.ams-ix.net/ams/documentation/general-terms-and-c... 3.3 Peering arrangements are not covered by these AMS-IX General Terms and Conditions or the AMS-IX Connection Agreement. The Customer is responsible for the negotiation, conclusion and implementation of peering arrangements with other users of the AMS-IX Infrastructure.
- cbg0 6y ago
  While you are correct, many networks connected have in fact an open peering policy: https://www.ams-ix.net/ams/connected-networks https://www.ams-ix.net/ams/connected-networks
  - gbil 6y ago
    sure and FRA IX also has a non-need for individual agreements policy https://www.de-cix.net/en/locations/germany/frankfurt https://www.de-cix.net/en/locations/germany/frankfurt but since I've seen the issues in my country in Europe where peering is not taken for granted, I wanted to make it visible that IXs are great yet open peering doesn't always come with them
xenadu02 6y ago
These used to be more common in the US and some still exist but the big telcos have all stopped upgrading their equipment which deliberately chokes the available bandwidth at those locations. They also deliberately don't on-board any new ones. This is the same strategy they used against Netflix if you recall - deliberately don't upgrade the linecards at choke points to ensure Netflix packets get dropped. Netflix eventually caved (and raised prices) so we're all paying for it on both ends: as ISP customers and Netflix subscribers. Who doesn't like a bit of double-dipping? They really really want to go back to the long-distance telephone days where people paid according to some kind of metering plan. That's the whole push behind getting rid of Net Neutrality (which is how the internet always worked up to that point - so much so we had to invent a new term to describe what was taken away). Why invest in new plant, fiber, or equipment? Just stop upgrading or upgrade very slowly and let increases in data usage dump money directly into your pocket. A great gig if you can get it.
- toast0 6y ago
  The reasons residential ISPs don't peer is two-fold. a) They're nearly all Tier-1 networks, and Tier-1 networks have very hard to meet peering requirements; AT&T, Verizon, and Sprint have been Tier-1 since forever, and now they're the dominant wireless carriers. AT&T and Verizon are also large residential ISPs, and CenturyLink is another, and they're a Tier-1 too. Comcast is effectively Tier-1 in the US, but they don't have an international network (yet?). b) residential connections are setup to be unbalanced flows, and peering policies at networks with restrictive policies generally require roughly balanced flows. Very few networks that do business with consumers are going to get balanced flows. Maybe a consumer oriented backup service can, if they mostly receive backups and don't send a lot of restores; 1-1 messaging without persistent server storage should be pretty much balanced, but nearly everything else is pretty much never going to be balanced. Of course, tier-1 networks also won't peer with you if you're a customer or have been a customer recently; so if you do end up with appropriate flows to your transit provider, you'd need to switch to another provider for some amount of time before coming back to ask for peering, and they may increase requirements in the meantime. Things were different when we had local/regional ISPs (dial-up and DSL line sharing). As those got big enough to justify multiple transit providers, they would gravitate towards peering places, because you can pickup multiple transit providers easily, and also peer to reduce costs for both networks when connecting to other networks at the peering place. A local ISP doesn't really care about traffic balance, they just want traffic off their transit link; and content networks don't care about traffic balance, they just want traffic off their transit link. If I had my way, we'd get mandatory line sharing for the last mile (and something for wireless?), so that we could have meaningful competition in IP networking at the local level again. Something where all the last mile infrastructure providers are prohibited from offering retail service would be best; they could keep their tier-1 ISP business, and their last mile business, but someone else would have to at least glue the two parts together.
traceroute66 6y ago
> you can basically reach every network in Europe by connecting to a few large exchanges for a fraction of what you'd pay for tier 1 transit Erm. No. No and, erm, NO ! Members at peering exchanges can be split into two categories "old boys club" and "everyone else". The former category are mean as mouse shit. They won't peer with "everyone else", in many cases they won't even peer with you even if you have a decent amount of traffic related to their AS. They certainly won't publish ANY routes on the exchange route servers. For example, BT (incumbent UK telco for those who don't know) are famously restrictive about who they will peer with at LINX. There is an old joke between LINX members, "Q. How do you know you've made it as an internet network ? A. BT agreed to peer with you over LINX". Basically the template answer when you ask BT for peering is "NO. Go away now you silly little man, you should be ashamed for asking." That's not to say European exchanges are worthless. They are great for interconnecting with other mid/small networks. They are great for connecting to CDNs, the hyperscalers, and the Facebooks, Googles etc. of this world.
- tiernano 6y ago
  in all fairness, the original op did say "basically"... I am peering in a few different IXes in Europe for personal stuff, mostly learning, and i get Apple, Google, Cloudflare, Facebook (even though i dont use them), Netflix and a lot of other places included... I do also agree about the whole "old boys club". My main ISP is Liberty Global (Virgin Media Ireland) and anyone i have talked to about getting a link with tells me its paid peering... would be handy, but too expensive... Also, they dont peer with Cloudflare direct either... Ahh well...
  - exikyut 6y ago
    Huh, that sounds interesting. How does this peering work? Is it mostly just a bunch of intangible configuration and routing, or do you actually have equipment racked in a colo that offers peering arrangements? I'm also curious how the pricing side of things works out. Sounds like it's not that expensive...
    - tiernano 6y ago
wut42 6y ago
It's a bit more complicated in reality. Most of the ISP consumers, even if present on the exchanges, aren't openly peering on it. It's great to interconnect with smaller networks and providers, but consumers ISP ? You have to either have a lot of traffic _they_ send to you, otherwise, peering isn't possible, or complicated, or you have to pay.

by cbg0 6y ago

sneak 6y ago

A big part of it is that simply connecting your own network to these exchanges is a lot more expensive in the US due to increased land size and corrupt legislation. Due to the intense regulations (which are bought and paid for at the legislative level, and mostly exist to protect the large incumbents like Verizon, AT&T, Comcast, CenturyLink, et al from any new competitors) you end up having to buy your "last mile" loop (which, due to the US's size and general car-centric design, is much more than just a mile in most cases) that connects your network (in your office, or data center, or whatever, over to the exchange point) from a small number of monopolists. They're thousands, or tens of thousands of dollars per month. "We don't care. We don't have to. We're the phone company."

gbil 6y ago

Internet Exchanges doesn't mean that a peering agreement is also in place automatically. For example: https://www.ams-ix.net/ams/documentation/general-terms-and-conditions https://www.ams-ix.net/ams/documentation/general-terms-and-c... 3.3 Peering arrangements are not covered by these AMS-IX General Terms and Conditions or the AMS-IX Connection Agreement. The Customer is responsible for the negotiation, conclusion and implementation of peering arrangements with other users of the AMS-IX Infrastructure.

cbg0 6y ago
While you are correct, many networks connected have in fact an open peering policy: https://www.ams-ix.net/ams/connected-networks https://www.ams-ix.net/ams/connected-networks
- gbil 6y ago
  sure and FRA IX also has a non-need for individual agreements policy https://www.de-cix.net/en/locations/germany/frankfurt https://www.de-cix.net/en/locations/germany/frankfurt but since I've seen the issues in my country in Europe where peering is not taken for granted, I wanted to make it visible that IXs are great yet open peering doesn't always come with them

xenadu02 6y ago

These used to be more common in the US and some still exist but the big telcos have all stopped upgrading their equipment which deliberately chokes the available bandwidth at those locations. They also deliberately don't on-board any new ones. This is the same strategy they used against Netflix if you recall - deliberately don't upgrade the linecards at choke points to ensure Netflix packets get dropped. Netflix eventually caved (and raised prices) so we're all paying for it on both ends: as ISP customers and Netflix subscribers. Who doesn't like a bit of double-dipping? They really really want to go back to the long-distance telephone days where people paid according to some kind of metering plan. That's the whole push behind getting rid of Net Neutrality (which is how the internet always worked up to that point - so much so we had to invent a new term to describe what was taken away). Why invest in new plant, fiber, or equipment? Just stop upgrading or upgrade very slowly and let increases in data usage dump money directly into your pocket. A great gig if you can get it.

toast0 6y ago
The reasons residential ISPs don't peer is two-fold. a) They're nearly all Tier-1 networks, and Tier-1 networks have very hard to meet peering requirements; AT&T, Verizon, and Sprint have been Tier-1 since forever, and now they're the dominant wireless carriers. AT&T and Verizon are also large residential ISPs, and CenturyLink is another, and they're a Tier-1 too. Comcast is effectively Tier-1 in the US, but they don't have an international network (yet?). b) residential connections are setup to be unbalanced flows, and peering policies at networks with restrictive policies generally require roughly balanced flows. Very few networks that do business with consumers are going to get balanced flows. Maybe a consumer oriented backup service can, if they mostly receive backups and don't send a lot of restores; 1-1 messaging without persistent server storage should be pretty much balanced, but nearly everything else is pretty much never going to be balanced. Of course, tier-1 networks also won't peer with you if you're a customer or have been a customer recently; so if you do end up with appropriate flows to your transit provider, you'd need to switch to another provider for some amount of time before coming back to ask for peering, and they may increase requirements in the meantime. Things were different when we had local/regional ISPs (dial-up and DSL line sharing). As those got big enough to justify multiple transit providers, they would gravitate towards peering places, because you can pickup multiple transit providers easily, and also peer to reduce costs for both networks when connecting to other networks at the peering place. A local ISP doesn't really care about traffic balance, they just want traffic off their transit link; and content networks don't care about traffic balance, they just want traffic off their transit link. If I had my way, we'd get mandatory line sharing for the last mile (and something for wireless?), so that we could have meaningful competition in IP networking at the local level again. Something where all the last mile infrastructure providers are prohibited from offering retail service would be best; they could keep their tier-1 ISP business, and their last mile business, but someone else would have to at least glue the two parts together.

traceroute66 6y ago

> you can basically reach every network in Europe by connecting to a few large exchanges for a fraction of what you'd pay for tier 1 transit Erm. No. No and, erm, NO ! Members at peering exchanges can be split into two categories "old boys club" and "everyone else". The former category are mean as mouse shit. They won't peer with "everyone else", in many cases they won't even peer with you even if you have a decent amount of traffic related to their AS. They certainly won't publish ANY routes on the exchange route servers. For example, BT (incumbent UK telco for those who don't know) are famously restrictive about who they will peer with at LINX. There is an old joke between LINX members, "Q. How do you know you've made it as an internet network ? A. BT agreed to peer with you over LINX". Basically the template answer when you ask BT for peering is "NO. Go away now you silly little man, you should be ashamed for asking." That's not to say European exchanges are worthless. They are great for interconnecting with other mid/small networks. They are great for connecting to CDNs, the hyperscalers, and the Facebooks, Googles etc. of this world.

tiernano 6y ago
in all fairness, the original op did say "basically"... I am peering in a few different IXes in Europe for personal stuff, mostly learning, and i get Apple, Google, Cloudflare, Facebook (even though i dont use them), Netflix and a lot of other places included... I do also agree about the whole "old boys club". My main ISP is Liberty Global (Virgin Media Ireland) and anyone i have talked to about getting a link with tells me its paid peering... would be handy, but too expensive... Also, they dont peer with Cloudflare direct either... Ahh well...
- exikyut 6y ago
  Huh, that sounds interesting. How does this peering work? Is it mostly just a bunch of intangible configuration and routing, or do you actually have equipment racked in a colo that offers peering arrangements? I'm also curious how the pricing side of things works out. Sounds like it's not that expensive...
  - tiernano 6y ago

wut42 6y ago

It's a bit more complicated in reality. Most of the ISP consumers, even if present on the exchanges, aren't openly peering on it. It's great to interconnect with smaller networks and providers, but consumers ISP ? You have to either have a lot of traffic _they_ send to you, otherwise, peering isn't possible, or complicated, or you have to pay.