4 ms·
This is trivially achievable with function calling, assuming the model you use supports this (which most models do at this point). Define a function `reportFac
by omneity 2y ago
This is trivially achievable with function calling, assuming the model you use supports this (which most models do at this point).
Define a function `reportFactual(isFactual: boolean)` and you will get standardized, machine-readable answers to do statistics with.
- codetrotter 2y agoI’ve used function calls with OpenAI. But are there any good local LLMs that you can run with Ollama that support function calling?
- omneity 2y agoIf you expect an OpenAI compatible API to use function calls, I don't think Ollama supports it yet (to be confirmed). However you can do it yourself using the appropriate tokens for the model. I know that Llama3, various Mistrals and Command-R support function calling out of the box. Here are the tokens to achieve this in Mixtral 8x22 https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1#function-calling-and-special-tokens https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1... Pass function definitions in the system prompt.
- indigodaddy 2y agoI think llamafile supports openai compatible api.. https://github.com/Mozilla-Ocho/llamafile https://github.com/Mozilla-Ocho/llamafile
- inimino 2y agoSimpler yet, just tell the model "Reply with 'Yes' or 'No'."