7 ms·

I read through a couple of these responses to the CMA by MS, Google and AWS and their smaller competitors as expected the hyperscalers refuse to acknowledge th

I read through a couple of these responses to the CMA by MS, Google and AWS and their smaller competitors as expected the hyperscalers refuse to acknowledge that the free ingress and expensive egress is a lock-in mechanism, and their smaller competitors complain bitterly about this the hyperscalers say they have to charge egress fees to pay for the costs in building their networks, but for some reason doesn't apply to ingress (which they're silent on) if they want to play this game then the CMA should simply make them charge the same for ingress and egress that way they can "fund their network costs" without issue, and if they want to make them both free then that's their decision

charcircuit 3y ago
>but for some reason doesn't apply to ingress (which they're silent on) The industry standard for peering is paying the 95th percentile of egress or ingress depending on whichever is greater. Ingress is free for these clouds because egress > ingress overall.
- blibble 3y ago
  I accept there's some level of cost, but the prices are so high it's hard to describe it as anything other than gouging to prevent competition
  - toast0 3y ago
    My personal feeling is they're moving costs around so that egress has a big margin and other items have a smaller or potentially negative margin. I've seen this at other providers. We did a competitive pricing exercise at my last company, and our overall cost went down, but the mechanism was per hosts costs went down significantly and egress costs went up significantly, and the per host cost decrease outweighed the egress cost increase. It still doesn't make sense to charge for ingress, because everybody knows that should be free, unless you're a residential ISP.
    - lokar 3y ago
      Many companies in many industries do this. It’s often simply not practical or sensible to price every SKU “fairly “ on some value or cost basis. Your margin varies from item to item (including negative) and the whole bundle works out.
    - ikiris 3y ago
      y'all vastly underestimate how much it costs to run a CDN as large as that at scale. bandwidth from cogent at whatever colo you can cross connect to them is wildly cheaper than "i need bandwith to everywhere" bandwidth.
- logifail 3y ago
  > The industry standard Pfft... > for peering is paying the 95th percentile of egress or ingress depending on whichever is greater. Ingress is free for these clouds because egress > ingress overall How about customers pay for actual usage, rather than some [fake] averaged-across-all-customers usage?
  - scottlamb 3y ago
    Why would the cloud provider charge for usage that doesn't actually cost them money? Unless usage patterns drastically change industry-wide, the ingress really doesn't matter to them. The egress does. It seems entirely reasonable to look more skeptically at cloud providers' exact charges vs cost for egress, particularly when high egress fees might contribute to lock-in, and when the public price sheet vs the preferred customer pricing might differ radically. But asking them to totally restructure the charges, inventing a charge for ingress when their actual total ingress cost is zero and, short of major industry-wide usage pattern changes, will remain zero? Why would you do that?
    - makomk 3y ago
      Yeah, it's long been common for more traditional hosting providers to limit egress traffic and charge overage fees for going over that but have no similar limit on incoming traffic for basically the same reason: their network-wide traffic patterns mean that egress is what costs them money and ingress is effectively free on top of that.
amluto 3y ago
> the hyperscalers say they have to charge egress fees to pay for the costs in building their networks, but for some reason doesn't apply to ingress (which they're silent on) This doesn’t pass the red face test IMO. The hyperscaler networks are indeed very expensive, but that’s because they need to provide non-blocking or near non-blocking performance within the availability zone, and the clouds don’t charge for this service. The Internet egress part ought to be straightforward on top of this: plug as much bandwidth worth of connections into the aforementioned extremely fancy internal network. Configure routes accordingly. It’s worth noting that the big clouds will sell you private links to your own facilities, and they charge for these links (which makes sense), but then they charge you for the traffic you send from inside the cloud to these links, which is absurd since they don’t charge for comparable traffic from inside the cloud to other systems in the same AZ.
- jeffbee 3y ago
  > they need to provide non-blocking or near non-blocking performance within the availability zone I see you've never tried GCP.
  - amluto 3y ago
    I have, but not for a use case where this matters. FWIW, Google has been working on these fancy nonblocking networks for a very long time. They’re very proud of them. Maybe they don’t actually use them for GCP, but Google definitely cares about network performance for their own purposes.
    - jeffbee 3y ago
      The whole concept of blocking is inapplicable to packet-switched networks. The whole time I was there I never heard anyone describe any of their several different types of networks as non-blocking. Indeed, the fact that they are centrally-controlled SDNs, where the control plane can tell any participant to stop talking to any other participant, seems to be logically the opposite of "non-blocking", if that circuit-switching terms were applicable. Your message seems to imply that these datacenter networks experience very little loss, and this is observably far from reality. In GCP you will observe levels of frame drops that a corporate datacenter architect would consider catastrophic.
TulliusCicero 3y ago
"Egress can be no more expensive than ingress" seems like a reasonable rule.
- qwertox 3y ago
  But not during normal operation when egress to consumers is in higher demand than ingress. This would raise ingress prices unnecessarily.
  - close04 3y ago
    For what the lock-in topic is concerned the expensive egress is just relevant when you want to leave your provider, not during daily operations with your consumers. Free ingress but prohibitively expensive egress is literally only there to make it as easy as possible for customers to come to the cloud provider and as hard as possible to leave. Coming to a provider or leaving them should cost the same or else it's just a trap. This is like Comcast giving you the initial installation for free but charging you an arm, a leg, and a year of your life to allow you to leave because "the process is costly". Both practices are scummy and abusive towards the customer.
mschuster91 3y ago
> the hyperscalers say they have to charge egress fees to pay for the costs in building their networks, but for some reason doesn't apply to ingress (which they're silent on) because ingress traffic volume is a fraction (in my experience, in website hosting of a well known household brand, barely 1%!) of egress traffic volume, and most peering connections are 1:1 in ingress/egress bandwidth so the egress bandwidth cost sets the price.
- NBJack 3y ago
  This also holds true on thebflip side for most consumer internet speeds (at least in the US). Many people have a x00 Mbps or even x Gbps downstream, but most have no more than x0 Mbps upstream. Literally their ability to pull traffic from websites is 50X in some cases than to push information out. Going beyond that (greater uploads) often costs significantly more. Whether or not these two are actually related isn't clear to me, but it is interesting.
  - womod 3y ago
    That's due to the DOCSIS standard for cable modems. They specced out more channels for downstream than upstream because of the limited bandwidth of copper and consumer priorities. With fiber there's an order of magnitude more bandwidth available, so the uneven split is much less (if at all?) common with the big backhaul lines between datacenters. For consumer fiber you'll usually get symmetric but for the most part it doesn't make sense as the vast majority of consumers just don't make use of their upstream bandwidth.

charcircuit 3y ago

>but for some reason doesn't apply to ingress (which they're silent on) The industry standard for peering is paying the 95th percentile of egress or ingress depending on whichever is greater. Ingress is free for these clouds because egress > ingress overall.

blibble 3y ago
I accept there's some level of cost, but the prices are so high it's hard to describe it as anything other than gouging to prevent competition
- toast0 3y ago
  My personal feeling is they're moving costs around so that egress has a big margin and other items have a smaller or potentially negative margin. I've seen this at other providers. We did a competitive pricing exercise at my last company, and our overall cost went down, but the mechanism was per hosts costs went down significantly and egress costs went up significantly, and the per host cost decrease outweighed the egress cost increase. It still doesn't make sense to charge for ingress, because everybody knows that should be free, unless you're a residential ISP.
  - lokar 3y ago
    Many companies in many industries do this. It’s often simply not practical or sensible to price every SKU “fairly “ on some value or cost basis. Your margin varies from item to item (including negative) and the whole bundle works out.
  - ikiris 3y ago
    y'all vastly underestimate how much it costs to run a CDN as large as that at scale. bandwidth from cogent at whatever colo you can cross connect to them is wildly cheaper than "i need bandwith to everywhere" bandwidth.
logifail 3y ago
> The industry standard Pfft... > for peering is paying the 95th percentile of egress or ingress depending on whichever is greater. Ingress is free for these clouds because egress > ingress overall How about customers pay for actual usage, rather than some [fake] averaged-across-all-customers usage?
- scottlamb 3y ago
  Why would the cloud provider charge for usage that doesn't actually cost them money? Unless usage patterns drastically change industry-wide, the ingress really doesn't matter to them. The egress does. It seems entirely reasonable to look more skeptically at cloud providers' exact charges vs cost for egress, particularly when high egress fees might contribute to lock-in, and when the public price sheet vs the preferred customer pricing might differ radically. But asking them to totally restructure the charges, inventing a charge for ingress when their actual total ingress cost is zero and, short of major industry-wide usage pattern changes, will remain zero? Why would you do that?
  - makomk 3y ago
    Yeah, it's long been common for more traditional hosting providers to limit egress traffic and charge overage fees for going over that but have no similar limit on incoming traffic for basically the same reason: their network-wide traffic patterns mean that egress is what costs them money and ingress is effectively free on top of that.

amluto 3y ago

> the hyperscalers say they have to charge egress fees to pay for the costs in building their networks, but for some reason doesn't apply to ingress (which they're silent on) This doesn’t pass the red face test IMO. The hyperscaler networks are indeed very expensive, but that’s because they need to provide non-blocking or near non-blocking performance within the availability zone, and the clouds don’t charge for this service. The Internet egress part ought to be straightforward on top of this: plug as much bandwidth worth of connections into the aforementioned extremely fancy internal network. Configure routes accordingly. It’s worth noting that the big clouds will sell you private links to your own facilities, and they charge for these links (which makes sense), but then they charge you for the traffic you send from inside the cloud to these links, which is absurd since they don’t charge for comparable traffic from inside the cloud to other systems in the same AZ.

jeffbee 3y ago
> they need to provide non-blocking or near non-blocking performance within the availability zone I see you've never tried GCP.
- amluto 3y ago
  I have, but not for a use case where this matters. FWIW, Google has been working on these fancy nonblocking networks for a very long time. They’re very proud of them. Maybe they don’t actually use them for GCP, but Google definitely cares about network performance for their own purposes.
  - jeffbee 3y ago
    The whole concept of blocking is inapplicable to packet-switched networks. The whole time I was there I never heard anyone describe any of their several different types of networks as non-blocking. Indeed, the fact that they are centrally-controlled SDNs, where the control plane can tell any participant to stop talking to any other participant, seems to be logically the opposite of "non-blocking", if that circuit-switching terms were applicable. Your message seems to imply that these datacenter networks experience very little loss, and this is observably far from reality. In GCP you will observe levels of frame drops that a corporate datacenter architect would consider catastrophic.

TulliusCicero 3y ago

"Egress can be no more expensive than ingress" seems like a reasonable rule.

qwertox 3y ago
But not during normal operation when egress to consumers is in higher demand than ingress. This would raise ingress prices unnecessarily.
- close04 3y ago
  For what the lock-in topic is concerned the expensive egress is just relevant when you want to leave your provider, not during daily operations with your consumers. Free ingress but prohibitively expensive egress is literally only there to make it as easy as possible for customers to come to the cloud provider and as hard as possible to leave. Coming to a provider or leaving them should cost the same or else it's just a trap. This is like Comcast giving you the initial installation for free but charging you an arm, a leg, and a year of your life to allow you to leave because "the process is costly". Both practices are scummy and abusive towards the customer.

mschuster91 3y ago

> the hyperscalers say they have to charge egress fees to pay for the costs in building their networks, but for some reason doesn't apply to ingress (which they're silent on) because ingress traffic volume is a fraction (in my experience, in website hosting of a well known household brand, barely 1%!) of egress traffic volume, and most peering connections are 1:1 in ingress/egress bandwidth so the egress bandwidth cost sets the price.

NBJack 3y ago
This also holds true on thebflip side for most consumer internet speeds (at least in the US). Many people have a x00 Mbps or even x Gbps downstream, but most have no more than x0 Mbps upstream. Literally their ability to pull traffic from websites is 50X in some cases than to push information out. Going beyond that (greater uploads) often costs significantly more. Whether or not these two are actually related isn't clear to me, but it is interesting.
- womod 3y ago
  That's due to the DOCSIS standard for cable modems. They specced out more channels for downstream than upstream because of the limited bandwidth of copper and consumer priorities. With fiber there's an order of magnitude more bandwidth available, so the uneven split is much less (if at all?) common with the big backhaul lines between datacenters. For consumer fiber you'll usually get symmetric but for the most part it doesn't make sense as the vast majority of consumers just don't make use of their upstream bandwidth.