4 ms·
Setting up Ollama via Docker was the easiest way for me to get up and running. Not 100% sure if it fits your constraints, but highly recommended.
by arcanemachiner 2y ago
Setting up Ollama via Docker was the easiest way for me to get up and running. Not 100% sure if it fits your constraints, but highly recommended.
- programd 2y agoAnother option is to download and compile llama.cpp and you should be able to run quantized models at an acceptable speed. https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp Also, if you can spend the $60 and buy another 32GB of RAM, this will allow you to run the 30GB models quite nicely.
- eth0up 2y agoUnfortunately motherboard is capped at 16Gb ram