3 ms·
> If you wanted to embed an LLM, you wouldn’t rely on a network endpoint and you could just use a llama.cpp binding. I do want to rely on a network endpoint, a
by _flux 2y ago
> If you wanted to embed an LLM, you wouldn’t rely on a network endpoint and you could just use a llama.cpp binding.
I do want to rely on a network endpoint, as not all my computers are capable of running LLMs locally in an efficient manner.
- wokwokwok 2y ago> If you wanted to embed an LLM…
- _flux 2y agoI read it as wanting to embed LLM functionality, but my reading might have been incorrect—and by the client library reference I understood it was referring to generic HTTP clients, but that reading might have also been incorrect :). Per the examples this looks like an LLM-over-network library and there doesn't seem to be too many of those available for Rust.