3 ms·
What would this need to be compatible with Ollama? I would love to run this with a local LLM!
by tacticalglare 2y ago
What would this need to be compatible with Ollama? I would love to run this with a local LLM!
- Smilinrobin 2y agoYou can already try other LLMs. Steps in docs.
- shriyansha 2y agoWe tried Gpt-4o, Anthropic haiku, Groq's 8b tool preview, Llama 3.2 8b tool support via Groq and all worked well. With Ollama in our machines, we didn't get good results maybe because of model limitation on function calling. We are experimenting with more models and smaller models so that running local with local models becomes possible. Although if you check the readme and example config file there is a way to give it a try with ollama.