4 ms·
My second link documents the `context` parameter for Ollama. It's unfortunately pretty hard to search for because the term is overloaded. `context` is a number
by lolinder 3y ago
My second link documents the `context` parameter for Ollama. It's unfortunately pretty hard to search for because the term is overloaded.
`context` is a number array that gets returned with each /generate request and can be returned to the server with the next request. There's not much more documentation than what's there in that link, but as I understand it the array is a JSON representation of the current state of inference at the time that Ollama returned. So when you send the context back up with a second request, Ollama can just load that array into memory and then start inference with the new prompt, rather than having to re-encode the whole conversation up to that point.
- CuriouslyC 3y agoI'm pretty sure it's just a cache of the context embedding.