4 ms·
ollama is a thin wrapper over llama.cpp, so i'd pose the opposite question - what does ollama give you over using llama.cpp directly?
by mappu 2y ago
ollama is a thin wrapper over llama.cpp, so i'd pose the opposite question - what does ollama give you over using llama.cpp directly?
- schappim 2y agoModel management, customisable HTTP APIs, monitoring, security features, "parallel requests" (batch processing), no requirement for HF auth etc...
- yjftsjthsd-h 2y agoEase of use. Rather like arduino or like docker vs chroot/jails/zones, there's nothing wrong with just using the underlying tech, but lowering friction has value.