4 ms·

Constellation, bringing AI to the Cloudflare stack

12 points by jgrahamc 3y ago

htrp 3y ago
looks like this is cpu bound models only?
hades32 3y ago
No mention of the resources that are available? The most interesting models currently need beefy GPUs, but I don't think Cloudflare has those...
NicoJuicy 3y ago
Some interesting examples, eg. The sentiment analysis in combination with email routing. I'm curious about the models and GPU support too. I'm guessing this is currently beta and they are awaiting popularity of this. The simple ( & cost-efficient) method would be to support CPU-first and then up the scale accordingly. No harm in that, but it could be made more clear regarding the roadmap. Edit: during private beta there is support for <10 mb. Models. https://developers.cloudflare.com/constellation/platform/data-model/#models https://developers.cloudflare.com/constellation/platform/dat... Discussions @ https://discord.gg/cloudflaredev https://discord.gg/cloudflaredev
SpaghettiX 3y ago
Its impossible to resize images or hash passwords on Cloudflare workers: the CPU time limit is too low. I don't see how this makes Cloudflare interesting for AI. I suspect the same problems will exist. This isn't so different than running onnx in a worker without constellation, and using KV directly? The negative is it's such a closed ecosystem.
- kentonv 3y ago
  > Its impossible to resize images or hash passwords on Cloudflare workers: the CPU time limit is too low. You're thinking of years ago when the CPU time limit was 50ms. These days the limit is 30 seconds, plenty of time for these tasks.
  - SpaghettiX 3y ago
    Thanks for your reply. I hit the CPU time limit a few months ago. I was paying for unbound and my account default was on. I guess I also hit the memory limit (128MB). Perhaps we should reply on: https://community.cloudflare.com/t/long-running-webcrypto-api/90953/31 https://community.cloudflare.com/t/long-running-webcrypto-ap... and https://community.cloudflare.com/t/options-for-password-hashing/138077 https://community.cloudflare.com/t/options-for-password-hash.... Though I'm not sure if it will work, certain algorithms might take more than 128MB.
  - SpaghettiX 3y ago
    Whilst you're here, when can we expect D1 to be distributed instead of a single instance, and when will D1's API to be stabilised? Thanks :) Edit: I just found https://blog.cloudflare.com/d1-turning-it-up-to-11/ https://blog.cloudflare.com/d1-turning-it-up-to-11/ so I'll read that first

12 points by jgrahamc 3y ago

htrp 3y ago

looks like this is cpu bound models only?

No mention of the resources that are available? The most interesting models currently need beefy GPUs, but I don't think Cloudflare has those...

NicoJuicy 3y ago

Some interesting examples, eg. The sentiment analysis in combination with email routing. I'm curious about the models and GPU support too. I'm guessing this is currently beta and they are awaiting popularity of this. The simple ( & cost-efficient) method would be to support CPU-first and then up the scale accordingly. No harm in that, but it could be made more clear regarding the roadmap. Edit: during private beta there is support for <10 mb. Models. https://developers.cloudflare.com/constellation/platform/data-model/#models https://developers.cloudflare.com/constellation/platform/dat... Discussions @ https://discord.gg/cloudflaredev https://discord.gg/cloudflaredev

SpaghettiX 3y ago

Its impossible to resize images or hash passwords on Cloudflare workers: the CPU time limit is too low. I don't see how this makes Cloudflare interesting for AI. I suspect the same problems will exist. This isn't so different than running onnx in a worker without constellation, and using KV directly? The negative is it's such a closed ecosystem.

kentonv 3y ago
> Its impossible to resize images or hash passwords on Cloudflare workers: the CPU time limit is too low. You're thinking of years ago when the CPU time limit was 50ms. These days the limit is 30 seconds, plenty of time for these tasks.
- SpaghettiX 3y ago
  Thanks for your reply. I hit the CPU time limit a few months ago. I was paying for unbound and my account default was on. I guess I also hit the memory limit (128MB). Perhaps we should reply on: https://community.cloudflare.com/t/long-running-webcrypto-api/90953/31 https://community.cloudflare.com/t/long-running-webcrypto-ap... and https://community.cloudflare.com/t/options-for-password-hashing/138077 https://community.cloudflare.com/t/options-for-password-hash.... Though I'm not sure if it will work, certain algorithms might take more than 128MB.
- SpaghettiX 3y ago
  Whilst you're here, when can we expect D1 to be distributed instead of a single instance, and when will D1's API to be stabilised? Thanks :) Edit: I just found https://blog.cloudflare.com/d1-turning-it-up-to-11/ https://blog.cloudflare.com/d1-turning-it-up-to-11/ so I'll read that first