3 ms·
Can't wait to see something like this merged into ollama (I'm sure there would be plenty of people fine-tuning models for it).
by rcarmo 3y ago
Can't wait to see something like this merged into ollama (I'm sure there would be plenty of people fine-tuning models for it).
- Me1000 3y agoOllama doesn't have their own inference engine, they just wrap llama.cpp. But yes, it will be awesome when it's more generally available.
- helloericsf 3y agoThe lab is tied to the vLLM project. I would say it might get picked up sooner by vLLM than other inference frameworks.