3 ms·
That's not bad at all! There is also Nx and Bumblebee in Elixir land - it really changes the how one approaches running models in production. The fact that one
by isodev 3y ago
That's not bad at all!
There is also Nx and Bumblebee in Elixir land - it really changes the how one approaches running models in production. The fact that one can put together a service (or local process) running any model published to hugging face in a couple of lines of code is amazing.
[0] https://github.com/elixir-nx/bumblebee/blob/main/examples/phoenix/text_classification.exs https://github.com/elixir-nx/bumblebee/blob/main/examples/ph...
[1] https://gist.github.com/toranb/8be408eaa97d5a5b795aec7d7fbee43e https://gist.github.com/toranb/8be408eaa97d5a5b795aec7d7fbee...
- anonyfox 3y agohow is the deployment story though? assuming a standard phoenix-on-fly.io process, I was under the assumption that the bumblebee models are downloaded at runtime? or are they "built" as part of the CI pipeline and then shipped within their docker container as blobs?
- isodev 3y agoThat’s all configurable. You can choose to download and build at startup, bundle into the docker image or “prebuild” the cache in advance / separately from the main app. I think it’s quite alright for both cloud, Docker and VPS-y deployments.
- anonyfox 3y agooh, interesting! to you have a link for me for the "bundle into docker" option?