3 ms·
PSA: DO NOT USE OLLAMA FOR TESTING. Ollama silently (!!!) drops messages if the context window is exceeded (instead of, you know, just erroring? who in the wor
by axiosgunnar 2y ago
PSA: DO NOT USE OLLAMA FOR TESTING.
Ollama silently (!!!) drops messages if the context window is exceeded (instead of, you know, just erroring? who in the world made this decision).
The workaround until now was to (not use ollama or) make sure to only send a single message. But now they seem to silently truncate single messages as well, instead of erroring! (this explains the sibling comment where a user could not reproduce the results locally).
Use LM Studio, llama.cpp, openrouter or anything else, but stay away from ollama!
- Technetium 2y agoI looked around to get confirmation, and I did find some related issues. Seems like it works properly when context is defined explicitly. There also appears to be a warning logged about "truncating input prompt", so it isn't an entirely silent failure. https://github.com/ollama/ollama/issues/2653 https://github.com/ollama/ollama/issues/2653 + https://github.com/ollama/ollama/issues/4967 https://github.com/ollama/ollama/issues/4967 + https://github.com/ollama/ollama/issues/7043 https://github.com/ollama/ollama/issues/7043 + https://github.com/ollama/ollama/issues/8144 https://github.com/ollama/ollama/issues/8144