4 ms·

TL;DR - author claims the hardest part of running a cloud PaaS is networking, thus the cloud providers charge you a ton for egress to outsource the optimization

TL;DR - author claims the hardest part of running a cloud PaaS is networking, thus the cloud providers charge you a ton for egress to outsource the optimization problem to you. It's not clear to me this claim is true - they're in the business of solving exactly this optimization problem - and they have, they're just capturing maximum profit. If it was really the hardest / most costly - then other providers wouldn't be able to offer it so much cheaper. It's clearly a solved problem. This is just the knob the pricing departments found to be the one to dial to juice profits to the max. The article seems like a puff piece to promote other providers. But it could be useful to people shopping for cheap egress.

zeroCalories 3y ago
??? This is obviously true. The bandwidth available in a geographic location is, practically speaking, limited. Roads are cheap to pave, but if there was congestion 24/7 and you paved all you could, then you would need to start rationing road usage. For a business the best way to ration is to charge more. Solving this problem requires a massive amount of engineering work, which would also justify the increased cost.
- onlyrealcuzzo 3y ago
  And you're not physically limited by how many machines you can put in a data center?
- weird-eye-issue 3y ago
  How does Cloudflare offer it for free?
  - withinboredom 3y ago
    Um. It’s not. They have a free plan, but it’s subsidized by their paying customers.
    - Syonyk 3y ago
      And a lot of their "good" features are only available to paying customers. Image resizing, Argo (fancy... routing aware something or another to your server), more than a few cache rules, and a range of other custom stuff. I think they do a really good job of it, personally. It's trivial to get your free account set up to play with and even host a pretty serious amount of "simple" traffic, and you can easily explore the features, see what's available for paid (they're quite transparent about the assorted costs which is nice), etc. It feels a bit like an honest version of Microsoft and Adobe's "look the other way for casual piracy, because you're familiar with the product..." of the late 90s and early 2000s. "Hey, try it out for free, and if the free stuff works for you, great. But if you need more, this is what else we offer!"
    - weird-eye-issue 3y ago
      I'm not saying Cloudflare is free I'm saying they don't charge bandwidth
      - withinboredom 3y ago
        They are eating the costs, as a marketing ploy. It won’t be free forever. They’ve said as much in their press releases. Basically, they are trying to pressure AWS and friends to lower their egress fees. Once they’re lowered, they’ll start charging for it.
        weird-eye-issue 3y ago
        Source? I'm incredibly skeptical that one day they will suddenly make a 180 and start charging for bandwidth
        withinboredom 3y ago
        I'd literally have to find the press releases from the day they announced R2, which kinda goes into depth on it a bit (and I can't find it). But logically, why wouldn't they do a 180? It has to be eating into their margins. Wholesale costs alone would estimate that it's eating 15-20% of their revenue. Remember when Uber used to be cheaper than a taxi? It's the same issue, different market; there is so much being subsidized by other money that if they ever run into any financial issues, hire the wrong executive, or simply get market dominance ... they'll shift this all to the customer.
- toast0 3y ago
  Bandwidth in a geographic location is certainly limited... there's only so many fiber strands available, and fiber transceivers only go up to a certain speed, +/- wave division; there's costs of transceivers and what not. But in most places where you'll find datacenters, the limit is rather high. It's also usually possible to run more fiber. If you take a look at the Seattle Internet Exchange [1] which is pretty public about their operations, they've got a router with 2x 36-port 400gbe line cards; three member facing 400gbe connections, 163 member facing 100gbe connections. That's a huge amount of bandwidth available at the exchange, some (perhaps much) of which is then delivered by the member to fiber that goes elsewhere. And SIX isn't the only peering location in the Seattle metro. SIX's traffic graph shows a peak of 2.8 Tbps; AMS-IX's graph shows a peak of 11.7 Tbps, AMS-IX doesn't offer 400 gbe though, but needing more than 100 gbps in one location is pretty rare. [1] https://www.seattleix.net/ https://www.seattleix.net/
  - sroussey 3y ago
    There is so much dark fiber out there. Bothering to activate it is a different story.
  - zeroCalories 3y ago
    There is a difference between being able to run more fiber, and being able to do it cheaply. The more you scale, the more you need everything you depend on to scale. For many places that calculation makes it not worth it.
- candiodari 3y ago
  Because of advancements in xWDM it is not limited. By replacing networking equipment (0.000001% of total cost) you can x10, x100, and more the bandwidth on an existing cable. The same (single mode) optical cable that was once capable of running 2mbps is now able to run 400 * 100Gbit. Now wireless is starting to repeat this evolution. This is one of the reasons for the old joke: "Do you know how to amass a small fortune in Telecom?" "Start with a big fortune"
- stefan_ 3y ago
  Bandwidth pricing works the same as product pricing, you are falling for the classic engineer trap in thinking the sum of the parts somehow determines the price. Cloud bandwidth prices are many many magnitudes more than what it "costs" to transfer.
api 3y ago
> author claims the hardest part of running a cloud PaaS is networking This is obviously false to me as someone who has done data center stuff and devops before. The hardest part is running complex services like databases, storage, and runtime environments like Kubernetes or serverless and doing a good enough job to deliver high uptime for a gigantic variety of customer workloads over which you have basically no control. The other hardest problem is security, because you are by definition running a ton of untrusted code on shared infrastructure. (scream emoji!) You can sort of tell software management is where the pain is because that's what keeps people on the cloud. That's the selling point. Without those managed services compute, storage, and bandwidth are ridiculously cheaper on VPS and bare metal providers. Networking is comparatively easy. It's mostly a game of over-engineering. You design your internal data center fabric to handle way more data than you think you will need. There was a time maybe a decade or two ago when networking at scale was a lot harder but today there are numerous off the shelf products designed for hyper-scale that you can just buy and deploy. This is part of why VPS and bare metal providers can sell bandwidth so cheap. You don't have to roll your own high performance fiber backplane fabric anymore.
inopinatus 3y ago
The author is just spitballing in their role as a Cloudflare shill. They don’t actually know jack shit about public cloud netops or pricing strategy. The so-called “bandwidth alliance” they spruik so hard is a negotiating vehicle for Cloudflare, intended to improve their position when haggling over peering, to support profitability of their DDoS protection service.
- kkielhofner 3y ago
  Where’s the profitability in the DDoS service? Even free accounts have had massive DDoS attacks mitigated - for free. There is no pricing anywhere in their portfolio for DDoS mitigation or bandwidth. Cloudflare sells a lot of products but DDoS protection isn’t one of them. It’s a hook to get users on the platform to potentially upsell later but if all you need is DDoS protection they provide it for free.
  - inopinatus 3y ago
    Enterprise DDoS protection is their goddamn flagship product. The Cloudflare website just says “call us”. That’s all. And it’s because E&G deals aren’t made through a website. Read the 10-K before spouting off about what value creation a middleman provides, and how it captures that value as revenue. Consumer services are last on their list and given little space. Large customers are explicitly stated as Cloudflare’s sales focus. And the entire top of the list of their disclosed competitors is everyone who sells a WAF or network protection apparatus to that segment. > they provide it for free This is naive. As ever, if it’s free, that means you’re part of the product. For social media, as is well understood, that’s your eyeballs sold to advertisers. In the case of Cloudflare, it’s consumer traffic that they pass as a middleman, used to build peering leverage through sheer volume. The clearest confirmation of this, however, is still Cloudflare going to all the time and trouble to wrangle together an industry collective - the so-called “bandwidth alliance” - for the express purpose of driving down settlement-based peering fees. They are not altruists. This is the architecture of their competitive advantage.
    - kkielhofner 3y ago

zeroCalories 3y ago

??? This is obviously true. The bandwidth available in a geographic location is, practically speaking, limited. Roads are cheap to pave, but if there was congestion 24/7 and you paved all you could, then you would need to start rationing road usage. For a business the best way to ration is to charge more. Solving this problem requires a massive amount of engineering work, which would also justify the increased cost.

onlyrealcuzzo 3y ago
And you're not physically limited by how many machines you can put in a data center?
weird-eye-issue 3y ago
How does Cloudflare offer it for free?
- withinboredom 3y ago
  Um. It’s not. They have a free plan, but it’s subsidized by their paying customers.
  - Syonyk 3y ago
    And a lot of their "good" features are only available to paying customers. Image resizing, Argo (fancy... routing aware something or another to your server), more than a few cache rules, and a range of other custom stuff. I think they do a really good job of it, personally. It's trivial to get your free account set up to play with and even host a pretty serious amount of "simple" traffic, and you can easily explore the features, see what's available for paid (they're quite transparent about the assorted costs which is nice), etc. It feels a bit like an honest version of Microsoft and Adobe's "look the other way for casual piracy, because you're familiar with the product..." of the late 90s and early 2000s. "Hey, try it out for free, and if the free stuff works for you, great. But if you need more, this is what else we offer!"
  - weird-eye-issue 3y ago
    I'm not saying Cloudflare is free I'm saying they don't charge bandwidth
    - withinboredom 3y ago
      They are eating the costs, as a marketing ploy. It won’t be free forever. They’ve said as much in their press releases. Basically, they are trying to pressure AWS and friends to lower their egress fees. Once they’re lowered, they’ll start charging for it.
      - weird-eye-issue 3y ago
        Source? I'm incredibly skeptical that one day they will suddenly make a 180 and start charging for bandwidth
        withinboredom 3y ago
        I'd literally have to find the press releases from the day they announced R2, which kinda goes into depth on it a bit (and I can't find it). But logically, why wouldn't they do a 180? It has to be eating into their margins. Wholesale costs alone would estimate that it's eating 15-20% of their revenue. Remember when Uber used to be cheaper than a taxi? It's the same issue, different market; there is so much being subsidized by other money that if they ever run into any financial issues, hire the wrong executive, or simply get market dominance ... they'll shift this all to the customer.
toast0 3y ago
Bandwidth in a geographic location is certainly limited... there's only so many fiber strands available, and fiber transceivers only go up to a certain speed, +/- wave division; there's costs of transceivers and what not. But in most places where you'll find datacenters, the limit is rather high. It's also usually possible to run more fiber. If you take a look at the Seattle Internet Exchange [1] which is pretty public about their operations, they've got a router with 2x 36-port 400gbe line cards; three member facing 400gbe connections, 163 member facing 100gbe connections. That's a huge amount of bandwidth available at the exchange, some (perhaps much) of which is then delivered by the member to fiber that goes elsewhere. And SIX isn't the only peering location in the Seattle metro. SIX's traffic graph shows a peak of 2.8 Tbps; AMS-IX's graph shows a peak of 11.7 Tbps, AMS-IX doesn't offer 400 gbe though, but needing more than 100 gbps in one location is pretty rare. [1] https://www.seattleix.net/ https://www.seattleix.net/
- sroussey 3y ago
  There is so much dark fiber out there. Bothering to activate it is a different story.
- zeroCalories 3y ago
  There is a difference between being able to run more fiber, and being able to do it cheaply. The more you scale, the more you need everything you depend on to scale. For many places that calculation makes it not worth it.
candiodari 3y ago
Because of advancements in xWDM it is not limited. By replacing networking equipment (0.000001% of total cost) you can x10, x100, and more the bandwidth on an existing cable. The same (single mode) optical cable that was once capable of running 2mbps is now able to run 400 * 100Gbit. Now wireless is starting to repeat this evolution. This is one of the reasons for the old joke: "Do you know how to amass a small fortune in Telecom?" "Start with a big fortune"
stefan_ 3y ago
Bandwidth pricing works the same as product pricing, you are falling for the classic engineer trap in thinking the sum of the parts somehow determines the price. Cloud bandwidth prices are many many magnitudes more than what it "costs" to transfer.

api 3y ago

> author claims the hardest part of running a cloud PaaS is networking This is obviously false to me as someone who has done data center stuff and devops before. The hardest part is running complex services like databases, storage, and runtime environments like Kubernetes or serverless and doing a good enough job to deliver high uptime for a gigantic variety of customer workloads over which you have basically no control. The other hardest problem is security, because you are by definition running a ton of untrusted code on shared infrastructure. (scream emoji!) You can sort of tell software management is where the pain is because that's what keeps people on the cloud. That's the selling point. Without those managed services compute, storage, and bandwidth are ridiculously cheaper on VPS and bare metal providers. Networking is comparatively easy. It's mostly a game of over-engineering. You design your internal data center fabric to handle way more data than you think you will need. There was a time maybe a decade or two ago when networking at scale was a lot harder but today there are numerous off the shelf products designed for hyper-scale that you can just buy and deploy. This is part of why VPS and bare metal providers can sell bandwidth so cheap. You don't have to roll your own high performance fiber backplane fabric anymore.

inopinatus 3y ago

The author is just spitballing in their role as a Cloudflare shill. They don’t actually know jack shit about public cloud netops or pricing strategy. The so-called “bandwidth alliance” they spruik so hard is a negotiating vehicle for Cloudflare, intended to improve their position when haggling over peering, to support profitability of their DDoS protection service.

kkielhofner 3y ago
Where’s the profitability in the DDoS service? Even free accounts have had massive DDoS attacks mitigated - for free. There is no pricing anywhere in their portfolio for DDoS mitigation or bandwidth. Cloudflare sells a lot of products but DDoS protection isn’t one of them. It’s a hook to get users on the platform to potentially upsell later but if all you need is DDoS protection they provide it for free.
- inopinatus 3y ago
  Enterprise DDoS protection is their goddamn flagship product. The Cloudflare website just says “call us”. That’s all. And it’s because E&G deals aren’t made through a website. Read the 10-K before spouting off about what value creation a middleman provides, and how it captures that value as revenue. Consumer services are last on their list and given little space. Large customers are explicitly stated as Cloudflare’s sales focus. And the entire top of the list of their disclosed competitors is everyone who sells a WAF or network protection apparatus to that segment. > they provide it for free This is naive. As ever, if it’s free, that means you’re part of the product. For social media, as is well understood, that’s your eyeballs sold to advertisers. In the case of Cloudflare, it’s consumer traffic that they pass as a middleman, used to build peering leverage through sheer volume. The clearest confirmation of this, however, is still Cloudflare going to all the time and trouble to wrangle together an industry collective - the so-called “bandwidth alliance” - for the express purpose of driving down settlement-based peering fees. They are not altruists. This is the architecture of their competitive advantage.
  - kkielhofner 3y ago