4 ms·
I want to like Ollama, but I wish it didn't obfuscate the actual directives (full prompt) that it sends to the underlying model. Ollama uses a custom templatizi
by vunderba 3y ago
I want to like Ollama, but I wish it didn't obfuscate the actual directives (full prompt) that it sends to the underlying model. Ollama uses a custom templatizing script in its Modelfiles to translate user input into the format that a specific model expects ([INST], etc), but it can be difficult to tell if it's working as expected because it won't show up in the logs at all.
Other than that it's a great project - very easy to get started and has a solid API implementation. I've got it running on both a Win 10 + WSL2 docker and on a Mac M1.
- coffeeri 3y agoYou can bypass the templating in raw mode, by setting the request parameter `raw` to true. https://github.com/jmorganca/ollama/blob/main/docs%2Fapi.md https://github.com/jmorganca/ollama/blob/main/docs%2Fapi.md
- vunderba 3y agoyeah I guess I could compare the output at 0.0 temperature with same prompt using Modelfile and then after using raw mode with my best guess of how the modelfile is creating the raw data it's passing to Ollama. I'd push a PR to the repo itself but I have zero experience with Go...