4 ms·
Yes everyone should just write cpp to call local LLMs obviously
by dangoodmanUT 1y ago
Yes everyone should just write cpp to call local LLMs obviously
- otabdeveloper4 1y agoYes, but llama.cpp already comes with a ready-made OpenAI-compatible inference server.
- reverius42 1y agoI think people are getting hung up on the "llama.cpp" name and thinking they need to write C++ code to use it. llama.cpp isn't (just) a C++ library/codebase -- it's a CLI application, server application (llama-server), etc.