4 ms·

Looking forward to GPU support for workers. Cloudflare announced they were working on it in 2021 [1], but it doesn't seem generally available yet; they still ha

Looking forward to GPU support for workers. Cloudflare announced they were working on it in 2021 [1], but it doesn't seem generally available yet; they still have a signup page for notification [2]. I know other companies have struggled with demand, so maybe they're doing it on an invite basis. [1]: https://blog.cloudflare.com/workers-ai/ https://blog.cloudflare.com/workers-ai/ [2]: https://www.cloudflare.com/nvidia-workers/ https://www.cloudflare.com/nvidia-workers/

jgrahamc 3y ago
Good things come to those who wait :-)
ushakov 3y ago
They probably just want to see how many people would sign up if they had it. Before they start building something nobody wants to pay for.
- babelfish 3y ago
  Agreed, but this is a blocker for anyone seriously considering using the product. CPU-only inference simply isn’t good enough for anything besides toy workloads. If they’re waiting for people to use Constellation before investing in GPU support, nobody will use constellation because it doesn’t have it, so they’ll never end up investing in it, and so on…
  - dknecht 3y ago
    I think people will be surprised how far CPU optimization will go for specialized inference. An example of the progress being made - http://ggml.ai/ http://ggml.ai/
  - NicoJuicy 3y ago
    They use it themselves to route traffic, which probably is a not gpu intensive. I've ran face detection on cpu and it takes 1 / secondv and it's probably a pretty intensive "action". Plenty of use-cases that don't require a gpu And plenty that require one too though.
NicoJuicy 3y ago
I was under the assumption that, because of how cloudflare works, it has to be globally available. That would mean support in every dc and a easily to expand capacity. They can't just make it available in one region and then see how it goes. I like how cloudflare works, but this use-case for them ( on the edge) seems more difficult to plan. It's not just the tech, it's the infrastructure in this case. Just my 2 cents

jgrahamc 3y ago

Good things come to those who wait :-)

ushakov 3y ago

They probably just want to see how many people would sign up if they had it. Before they start building something nobody wants to pay for.

babelfish 3y ago
Agreed, but this is a blocker for anyone seriously considering using the product. CPU-only inference simply isn’t good enough for anything besides toy workloads. If they’re waiting for people to use Constellation before investing in GPU support, nobody will use constellation because it doesn’t have it, so they’ll never end up investing in it, and so on…
- dknecht 3y ago
  I think people will be surprised how far CPU optimization will go for specialized inference. An example of the progress being made - http://ggml.ai/ http://ggml.ai/
- NicoJuicy 3y ago
  They use it themselves to route traffic, which probably is a not gpu intensive. I've ran face detection on cpu and it takes 1 / secondv and it's probably a pretty intensive "action". Plenty of use-cases that don't require a gpu And plenty that require one too though.

NicoJuicy 3y ago

I was under the assumption that, because of how cloudflare works, it has to be globally available. That would mean support in every dc and a easily to expand capacity. They can't just make it available in one region and then see how it goes. I like how cloudflare works, but this use-case for them ( on the edge) seems more difficult to plan. It's not just the tech, it's the infrastructure in this case. Just my 2 cents