3 ms·
I’ve used function calls with OpenAI. But are there any good local LLMs that you can run with Ollama that support function calling?
by codetrotter 2y ago
I’ve used function calls with OpenAI. But are there any good local LLMs that you can run with Ollama that support function calling?
- omneity 2y agoIf you expect an OpenAI compatible API to use function calls, I don't think Ollama supports it yet (to be confirmed). However you can do it yourself using the appropriate tokens for the model. I know that Llama3, various Mistrals and Command-R support function calling out of the box. Here are the tokens to achieve this in Mixtral 8x22 https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1#function-calling-and-special-tokens https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1... Pass function definitions in the system prompt.
- indigodaddy 2y agoI think llamafile supports openai compatible api.. https://github.com/Mozilla-Ocho/llamafile https://github.com/Mozilla-Ocho/llamafile