4 ms·
Llama.cpp is a lot of C SIMD. I think using C++ elsewhere in the project just made sense in its infancy, when it was CPU only. Rust is really interesting for s
by brucethemoose2 3y ago
Llama.cpp is a lot of C SIMD. I think using C++ elsewhere in the project just made sense in its infancy, when it was CPU only.
Rust is really interesting for stuff besides running the actual llm though. Python can be a huge performance/debugging pain when your frontend and such get huge.
- jart 3y ago> Rust is really interesting for stuff besides running the actual llm though. That's the reason why llama.cpp is so attractive though. It just runs the LLM.
- brucethemoose2 3y agoThats not giving the project enough credit though, it implements a ton of very useful features outside the llm itself.