3 ms·
Do the models run in a docker container? Probably one of my favorite things about Ollama.
by syntaxing 3y ago
Do the models run in a docker container? Probably one of my favorite things about Ollama.
- mchiang 3y agodisclaimer: I'm one of the maintainers working on Ollama. I would love to hear how you are using Ollama. One of the upcoming releases will involve an official release of Ollama on Linux with CUDA support of the box. From there, we will publish Ollama Docker images to enable GPU support as well.
- syntaxing 3y agoThank you for your amazing work! I more or less use it as a llama cpp replacement cause I honestly can’t figure out a good prompt structure so I get bad results. On my M1 MBP, I use it for stuff that can’t go to GPT but can only use upto 13B. I also run a 34B model with CUBLAS enabled on my Linux server by modifying the go files mentioned in one of the issue tickets.
- cztomsik 3y agoSorry, I thought I have answered already. No, everything runs as one native-like (mix of native and webview) application.