2 ms·
You'll want to try llama.cpp [1]. The set of models that it can support is expanding [2]. Folks have also written services [3] that wrap around it. [1] https:/
by curious_cat_163 3y ago
You'll want to try llama.cpp [1]. The set of models that it can support is expanding [2]. Folks have also written services [3] that wrap around it.
[1] https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp
[2] https://huggingface.co/TheBloke https://huggingface.co/TheBloke
[3] https://github.com/abetlen/llama-cpp-python https://github.com/abetlen/llama-cpp-python