4 ms·
Yeah it's great, you can get this running in no time (even without gpu, just to play with it): docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ol
by imp0cat 3y ago
Yeah it's great, you can get this running in no time (even without gpu, just to play with it):
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
docker exec -it ollama ollama run llama2
And that's all!
- stefandesu 3y agoIt's nice to have this possibility on Linux now. It wasn't much more complicated on macOS before: brew install ollama brew services start ollama ollama run llama2 Works great on my M1 MacBook Air, although it's definitely not as good as ChatGPT.
- politelemon 3y agoAnd with GPU just do this for the first command docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama You'll need to have the NVidia container toolkit installed.
- alexchantavy 3y agoThank you! For docker-compose, do deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: [gpu] Docs: https://docs.docker.com/compose/gpu-support/ https://docs.docker.com/compose/gpu-support/