4 ms·

I don't understand. When you run `ollama pull` (or even `ollama run` which will pull if you don't have the model), that is using docker is it not? The code even

• _ea1k 3y ago
  ollama maintains a registry of ollama compatible llms. I don't see any indication that they use docker for that registry, but I could have overlooked something. I also don't see any docker/podman containers running when using ollama.
• hobofan 3y ago
  I haven't looked to deeply into the code, but it looks to me like it just uses OCI/Docker images as a distribution format. You can find some evidence of that in the model directory (mentioned in the FAQ[0]), where you can find manifest files that have contents of a docker manifest. Using this format likely makes distribution a lot easier as they can just use an existing OCI registry for hosting the images, and you can use existing push/pull primitives. OCI images are often used nowadays for things besides docker images if you have a format (like a layered filesystem) that is nicely content-addressable. The default registry for Ollama is https://ollama.ai/library https://ollama.ai/library (I don't know about the API endpoint). I'm not sure if/how you can actually use it with another registry endpoint. When people say that something "uses Docker" that usually involves utilizing the Docker daemon with it's sandboxing, networking, etc.. Ollama doesn't do or need to do that as it can just do inference natively based on GGUF files (+ the other configuration that comes as part of their model file). However, as you have noticed, it does borrow heavily from Docker on a conceptual level. E.g. apart from the registry mechanism its Modelfile heavily mimics a Dockerfile. [0]: https://github.com/jmorganca/ollama/blob/aabd71aede99443d5858213eec4e8d4c10713c55/docs/faq.md#where-are-models-stored https://github.com/jmorganca/ollama/blob/aabd71aede99443d585...