3 ms·
llama.cpp is open source and community-run, and is probably the most popular implementation for locally hosted models right now. That aside, in general, the qu
by int_19h 2y ago
llama.cpp is open source and community-run, and is probably the most popular implementation for locally hosted models right now.
That aside, in general, the question is - why bother so long as the libraries are open source and good enough? They can always be forked if there are any shenanigans, but why do so proactively? FWIW llama.cpp originally showed up as an answer to a very specific question (fast inference on Apple Silicon) and then just grew organically from there. Similarly, you can expect other libraries to fill the niches that don't already have "good enough" solutions.