3 ms·
I’ve been wondering, without stuff like llama.cpp and ollama etc how exactly were you intended to run Mistral when they released it?
by pram 3y ago
I’ve been wondering, without stuff like llama.cpp and ollama etc how exactly were you intended to run Mistral when they released it?
- tarruda 3y agoThe main framework for running LLMs was pytorch, which AFAIK has a slow CPU implementation, so only GPU/CUDA users would be covered. Llama.cpp was the project that popularized running LLMs on the CPU due to its very efficient implementation. Ollama is a frontend to it.