4 ms·

Could this work with llama.cpp, since it’s the engine behind Ollama? I usually build llama.cpp from source and download quantized (GGUF) models from Huggingfac

by car 2y ago

Could this work with llama.cpp, since it’s the engine behind Ollama? I usually build llama.cpp from source and download quantized (GGUF) models from Huggingface, haven’t used Ollama this far.

Dontizi 2y ago
No, for now, I’ve only made it work with Ollama, but it could be ideal to do it directly on llama.cpp. Thank you, I’ll take note of it.
- car 2y ago
  That would be great. Llama.cpp’s built in server offers HTTP embedding endpoints.