kiview

4 ms·

1.
by kiview 8mo ago
Yeah, we are on it. In the current version, things are hardcoded and implicit (we are also in experimental preview), but soon it will be configurable and explicit.
2.
by kiview 1y ago
You can run the same model in production, using Docker Model Runner in Docker CE. This is assuming, the model works for your production use case :)
3.
by kiview 1y ago
You can check out my blog post on publishing models to Docker Hub using Docker Model Runner :) https://www.docker.com/blog/publish-ai-models-on-docker-hub/
4.
by kiview 1y ago
Unfortunately not, since the container won't have access to the Apple silicon GPU. That's why in our architecture, we have to run llama.cpp as a host process and wire it up with the rest of the Docker Desktop architecture, to make
5.
by kiview 1y ago
(disclaimer: I'm leading the Docker Model Runner team at Docker) We decided to start with Apple silicon Macs, because they provide one of the worst experiences of running LLMs in a containerized form, while at the same time having very
6.
by kiview 1y ago
Better late than never ;)
7.
by kiview 1y ago
(disclaimer: I'm leading the Docker Model Runner team at Docker) We are not packaging models as Docker images, since indeed that is the wrong fit and comes with all kinds of technical problems. It also feels wrong to pure package data
8.
by kiview 1y ago
(disclaimer: I'm leading the Docker Model Runner team at Docker) It's fine to disagree of course, but we envision Docker as a tool that has a higher abstraction level than just container management. That's why having a new do
9.
by kiview 1y ago
(disclaimer: I'm leading the Docker Model Runner team at Docker) You were always able to manually install Docker CE within WSL2 on Windows. But if you want to have an integrated Docker experience on the Windows host, you need to use Do