3 ms·
> Btw, you can also run Mistral locally within the Docker model runner on a Mac. Efficiently? I thought macOS does not have API so that Docker could use GPU.
by nicce 1y ago
> Btw, you can also run Mistral locally within the Docker model runner on a Mac.
Efficiently? I thought macOS does not have API so that Docker could use GPU.
- jt_b 1y agoI haven't/wouldn't use it because I have a decent K8S ollama/open-webui setup, but docker announced this a month ago: https://www.docker.com/blog/introducing-docker-model-runner https://www.docker.com/blog/introducing-docker-model-runner
- nicce 1y agoHmm, I guess that is not actually running inside container/ there is no isolation. Some kind of new way that mixes llama.cpp , OCI format and docker CLI.