4 ms·
The main framework for running LLMs was pytorch, which AFAIK has a slow CPU implementation, so only GPU/CUDA users would be covered. Llama.cpp was the project
by tarruda 3y ago
The main framework for running LLMs was pytorch, which AFAIK has a slow CPU implementation, so only GPU/CUDA users would be covered.
Llama.cpp was the project that popularized running LLMs on the CPU due to its very efficient implementation. Ollama is a frontend to it.