4 ms·
Compose Spec Updated with <models> for AI Workloads
- mikesir87 1y agoExcited to see models get first-class support! Will be interesting to see what opportunities this opens with this being formalized (other tooling/integrations, etc.). Note that Compose files using the previous model declarations (using providers) will still work.
- pploug 1y agoThe big question is still - how will the models reach production?
- kiview 1y agoYou can check out my blog post on publishing models to Docker Hub using Docker Model Runner :) https://www.docker.com/blog/publish-ai-models-on-docker-hub/ https://www.docker.com/blog/publish-ai-models-on-docker-hub/
- pploug 1y agoWell, more wondering how I go from having local models into a production app with the model available - I guess swap it with a cloud offering?
- kiview 1y agoYou can run the same model in production, using Docker Model Runner in Docker CE. This is assuming, the model works for your production use case :)
- asuarezfr 1y agoThat sounds interesting
- djdto 1y agoFollowing this.
- AsmodiusVI 1y agoJust saw this in the release notes, super excited to try it out. Compose supporting an AI agent stack out of the box could be a big deal. I just saw a Reddit thread around using compose in production and I'm thinking I can put these two together. The idea of spinning up a whole multi-container setup for an LLM app (like a vector DB, orchestrator, backend, etc.) with a single compose up could work to actually get stuff to prod. I’ve been hacking around and struggling with this. Curious to see how far I can push this, might finally be the clean setup I can share.
- deleted 1y ago[deleted]
- pianoswim 1y agosuper pumped to see models get this support through docker
- nunez 1y agoThis is brilliant.