4 ms·

Some more on how we think about edge computing and the serverless market: https://blog.cloudflare.com/cloudflare-workers-serverless-week/ https://blog.cloudflar

Some more on how we think about edge computing and the serverless market: https://blog.cloudflare.com/cloudflare-workers-serverless-week/ https://blog.cloudflare.com/cloudflare-workers-serverless-we...

anonfunction 6y ago
Any plans to offer GPUs for Workers?
- majke 6y ago
  What is your use case? In my experience it's hard to use them properly. I failed to use modern GPU's (V100's) effectively for even large computing tasks. I found good compiler producing AVX2 on many cores to be easier and cheaper.
  - anonfunction 6y ago
    We're using GPUs already on AWS for inference. The big problem is we have a worker pool of GPU servers and a task queue, leading to some latency if we don't have enough servers running and by the time the EC2 auto scaling group adds one the queue could be drained. Also we are seeing extra latency from requests originating from Asia since we currently have our servers located in a US region. So basically all the classic benefits of serverless and edge computing would apply, the difference being we would need some more drivers, python libraries and a GPU attached.
rewq4321 6y ago
Any insight into why bandwidth is so expensive?
- mrkurt 6y ago
  If you're selling hosting, you can either sell CPU at close to cost and make money on bandwidth, or make your margins on CPU. If people are building things that don't stress the CPUs at all, you can load up servers and make ~$60-70/mo margins on CPUs. When they expect dedicated CPUs and memory, they're suddenly going to price shop across cloud providers and won't pay enough to make those same margins.
- eastdakota 6y ago
  Bandwidth is one of the things that gets significantly cheaper at scale. For most users, if they call a wholesale bandwidth provider the rates they get will be more than what a public cloud provider will offer. So it looks like a good deal. That's much more true than with something like hardware/compute where the differences between what you can buy a server for and what a big cloud provider can is less extreme. That's not to say Amazon or Google can't get better deals on servers than someone who just phones up Dell, but it's nothing like the savings you can get at scale on bandwidth. What's been interesting is how little competition there has been between the different public clouds on bandwidth pricing. That's starting to change and I'm proud of how we've pushed that at Cloudflare with things like the Bandwidth Alliance (https://www.cloudflare.com/bandwidth-alliance/ https://www.cloudflare.com/bandwidth-alliance/). Oracle's cloud is also pushing on this (see Zoom's comments on why they chose the Oracle cloud). If you saw the margins that AWS gets on bandwidth you'd throw up in your mouth.
  - MaxBarraclough 6y ago
    > If you saw the margins that AWS gets on bandwidth you'd throw up in your mouth. Presumably the thinking behind this approach is that it allows them to advertise cheap instance costs and storage costs, while still making bank, assuming the customer's traffic volumes are non-negotiable? Also has the effect of lowering prices for casual dabblers, who might want to play about with what's on offer without moving much traffic.
  - jorams 6y ago
    I find this comment rather confusing under an announcement for Workers Unbound, which charges $0.09 per GB. That's AWS/GCP/Azure-level pricing, and definitely does not push competition on that front.
  - CodesInChaos 6y ago
    > If you saw the margins that AWS gets on bandwidth you'd throw up in your mouth. Assuming that Hetzner's 1 EUR/TB isn't losing them money, AWS's 50-90 USD/TB should put the margin at 98% plus.
ArtWomb 6y ago
>>> Whether you think that regulations that appear to require local data storage and processing are a good idea or not — and I personally think they are bad policies that will stifle innovation — my sense is the momentum behind them is significant enough that they are, at this point, likely inevitable Agree it's stealth protectionism and security overreach. But do data sovereignty laws really form the basis of an inevitable future federated internet? Trade agreements like CPPTPA and USAMCA seem to prohibit. Or restrict sensitive finance or health data. And most users of consumer internet apps like TikTok aren't too concerned. They just want fresh memes ;) Just signed up to Cloudflare Unbounded private beta (feel free to fast track). And thanks for all your hard work. $NET has been a stand out performer!
- ocdtrekkie 6y ago
  The Internet was always supposed to be federated/decentralized. The idea of megacorps centralizing all of your data isn't the Internet. And the fact that we live in the real world and the Internets servers exist in countries subject to various laws, there was no way a global Internet was going to ever be without any sort of borders. The idea that the Internet is universally global and where your data is doesn't matter is idealistic fiction, not reality.
  - manigandham 6y ago
    There's plenty of space between the extremes of centralized data and being stored in the same country as the user. Freedom to store and process in more logical, secure and efficient places is a good thing.
    - ocdtrekkie 6y ago
      Your user will almost always benefit from the data being stored in the same country as them. Using an app across an ocean tends to suck. And the user gets to benefit from their country's consumer protection laws. Which is to say, the only parties that benefit from being able to store your data in other countries is often global monopolies shifting your data to where they have the least legal oversight.
harpratap 6y ago
What do you think about 5G vendors being able to also provide edge compute, which is more "edgy" than any CDN right now ?(they are literally going to be on every single block or street)
- daxfohl 6y ago
  While 5G receivers will be on every block, nobody is going to be deploying custom code on the receivers. It's about a 10 ms round trip from there to the metro hub, where custom code will be running. (So roughly equivalent to CDN for wired/wifi connections).
  - harpratap 6y ago
    > nobody is going to be deploying custom code on the receivers Why do you say so? There's nothing technically stopping a teleco to add extra nodes there, use 10% for themselves and rent out the remaining 90% resources.
    - daxfohl 6y ago
      Technically, maybe, but telco's are not interested in that business. They know nothing about hosting user code, have no way to support a gazillion instances of random customers running random code on a gazillion nodes around the world in heat, rain, ice, whatever. How would you stand a chance doing customer support on that? There's also security; they can't just store/process customer certificates on minimally secured boxes on street corners. Just about any useful code requires some kind of cert or secret or connection string somewhere. Cloud providers require all sorts of physical security restrictions, guards, cameras, sharks with lasers and such in order to host these things. The way it's working now is telcos are outsourcing it to the cloud providers, and cloud providers are sticking racks in the carriers' metro hubs (with shark/laser requirements). These racks show up like mini regions in cloud providers' portals, so users can deploy VM's there just like any other region. Much lower barrier to entry that way. Maybe someday you'll get sub-microsecond latency on custom apps from some box hanging on your street, but that's not coming anytime soon AFAIK.
deleted 6y ago
[deleted]

anonfunction 6y ago

Any plans to offer GPUs for Workers?

majke 6y ago
What is your use case? In my experience it's hard to use them properly. I failed to use modern GPU's (V100's) effectively for even large computing tasks. I found good compiler producing AVX2 on many cores to be easier and cheaper.
- anonfunction 6y ago
  We're using GPUs already on AWS for inference. The big problem is we have a worker pool of GPU servers and a task queue, leading to some latency if we don't have enough servers running and by the time the EC2 auto scaling group adds one the queue could be drained. Also we are seeing extra latency from requests originating from Asia since we currently have our servers located in a US region. So basically all the classic benefits of serverless and edge computing would apply, the difference being we would need some more drivers, python libraries and a GPU attached.

rewq4321 6y ago

Any insight into why bandwidth is so expensive?

mrkurt 6y ago
If you're selling hosting, you can either sell CPU at close to cost and make money on bandwidth, or make your margins on CPU. If people are building things that don't stress the CPUs at all, you can load up servers and make ~$60-70/mo margins on CPUs. When they expect dedicated CPUs and memory, they're suddenly going to price shop across cloud providers and won't pay enough to make those same margins.
eastdakota 6y ago
Bandwidth is one of the things that gets significantly cheaper at scale. For most users, if they call a wholesale bandwidth provider the rates they get will be more than what a public cloud provider will offer. So it looks like a good deal. That's much more true than with something like hardware/compute where the differences between what you can buy a server for and what a big cloud provider can is less extreme. That's not to say Amazon or Google can't get better deals on servers than someone who just phones up Dell, but it's nothing like the savings you can get at scale on bandwidth. What's been interesting is how little competition there has been between the different public clouds on bandwidth pricing. That's starting to change and I'm proud of how we've pushed that at Cloudflare with things like the Bandwidth Alliance (https://www.cloudflare.com/bandwidth-alliance/ https://www.cloudflare.com/bandwidth-alliance/). Oracle's cloud is also pushing on this (see Zoom's comments on why they chose the Oracle cloud). If you saw the margins that AWS gets on bandwidth you'd throw up in your mouth.
- MaxBarraclough 6y ago
  > If you saw the margins that AWS gets on bandwidth you'd throw up in your mouth. Presumably the thinking behind this approach is that it allows them to advertise cheap instance costs and storage costs, while still making bank, assuming the customer's traffic volumes are non-negotiable? Also has the effect of lowering prices for casual dabblers, who might want to play about with what's on offer without moving much traffic.
- jorams 6y ago
  I find this comment rather confusing under an announcement for Workers Unbound, which charges $0.09 per GB. That's AWS/GCP/Azure-level pricing, and definitely does not push competition on that front.
- CodesInChaos 6y ago
  > If you saw the margins that AWS gets on bandwidth you'd throw up in your mouth. Assuming that Hetzner's 1 EUR/TB isn't losing them money, AWS's 50-90 USD/TB should put the margin at 98% plus.

ArtWomb 6y ago

>>> Whether you think that regulations that appear to require local data storage and processing are a good idea or not — and I personally think they are bad policies that will stifle innovation — my sense is the momentum behind them is significant enough that they are, at this point, likely inevitable Agree it's stealth protectionism and security overreach. But do data sovereignty laws really form the basis of an inevitable future federated internet? Trade agreements like CPPTPA and USAMCA seem to prohibit. Or restrict sensitive finance or health data. And most users of consumer internet apps like TikTok aren't too concerned. They just want fresh memes ;) Just signed up to Cloudflare Unbounded private beta (feel free to fast track). And thanks for all your hard work. $NET has been a stand out performer!

ocdtrekkie 6y ago
The Internet was always supposed to be federated/decentralized. The idea of megacorps centralizing all of your data isn't the Internet. And the fact that we live in the real world and the Internets servers exist in countries subject to various laws, there was no way a global Internet was going to ever be without any sort of borders. The idea that the Internet is universally global and where your data is doesn't matter is idealistic fiction, not reality.
- manigandham 6y ago
  There's plenty of space between the extremes of centralized data and being stored in the same country as the user. Freedom to store and process in more logical, secure and efficient places is a good thing.
  - ocdtrekkie 6y ago
    Your user will almost always benefit from the data being stored in the same country as them. Using an app across an ocean tends to suck. And the user gets to benefit from their country's consumer protection laws. Which is to say, the only parties that benefit from being able to store your data in other countries is often global monopolies shifting your data to where they have the least legal oversight.

harpratap 6y ago

What do you think about 5G vendors being able to also provide edge compute, which is more "edgy" than any CDN right now ?(they are literally going to be on every single block or street)

daxfohl 6y ago
While 5G receivers will be on every block, nobody is going to be deploying custom code on the receivers. It's about a 10 ms round trip from there to the metro hub, where custom code will be running. (So roughly equivalent to CDN for wired/wifi connections).
- harpratap 6y ago
  > nobody is going to be deploying custom code on the receivers Why do you say so? There's nothing technically stopping a teleco to add extra nodes there, use 10% for themselves and rent out the remaining 90% resources.
  - daxfohl 6y ago
    Technically, maybe, but telco's are not interested in that business. They know nothing about hosting user code, have no way to support a gazillion instances of random customers running random code on a gazillion nodes around the world in heat, rain, ice, whatever. How would you stand a chance doing customer support on that? There's also security; they can't just store/process customer certificates on minimally secured boxes on street corners. Just about any useful code requires some kind of cert or secret or connection string somewhere. Cloud providers require all sorts of physical security restrictions, guards, cameras, sharks with lasers and such in order to host these things. The way it's working now is telcos are outsourcing it to the cloud providers, and cloud providers are sticking racks in the carriers' metro hubs (with shark/laser requirements). These racks show up like mini regions in cloud providers' portals, so users can deploy VM's there just like any other region. Much lower barrier to entry that way. Maybe someday you'll get sub-microsecond latency on custom apps from some box hanging on your street, but that's not coming anytime soon AFAIK.

deleted 6y ago

[deleted]