23 ms·

Yes, I agree. I've just ran the model locally and it's making a good impression. I've tested it with some ruby/rspec gotchas, which it handled nicely. I'll giv

• ericb 1y ago
  In ollama, how do you set up the larger context, and figure out what settings to use? I've yet to find a good guide. I'm also not quite sure how I should figure out what those settings should be for each model. There's context length, but then, how does that relate to input length and output length? Should I just make the numbers match? 32k is 32k? Any pointers?
  • lis 1y ago
    For aider and ollama, see: https://aider.chat/docs/llms/ollama.html https://aider.chat/docs/llms/ollama.html Just for ollama, see: https://github.com/ollama/ollama/blob/main/docs/faq.md#how-can-i-specify-the-context-window-size https://github.com/ollama/ollama/blob/main/docs/faq.md#how-c... I’m using llama.cpp though, so I can’t confirm these methods.
    • nico 1y ago
      Are you using it with aider? If so, how has your experience been?
  • zackify 1y ago
    Ollama breaks for me. If I manually set the context higher. The next api call from clone resets it back. And ollama keeps taking it out of memory every 4 minutes. LM studio with MLX on Mac is performing perfectly and I can keep it in my ram indefinitely. Ollama keep alive is broken as a new rest api call resets it after. I’m surprised it’s this glitched with longer running calls and custom context length.