5 ms·

The big question - which one of these new agents can consume local models to a reasonable degree? I would like to ditch the dependency on external APIs - willin

The big question - which one of these new agents can consume local models to a reasonable degree? I would like to ditch the dependency on external APIs - willing to trade some performance in lieu.

deleted 1y ago
[deleted]
jasonm23 1y ago
Crush has an open issue (2 weeks) to add Ollama support - it's in progress.
- Aperocky 1y ago
  nice, that would be my reason to use Crush.
  - tempodox 1y ago
    Me too.
- ggerganov 1y ago
  They should add "custom endpoint" support instead [0]. [0] https://github.com/microsoft/vscode/issues/249605 https://github.com/microsoft/vscode/issues/249605
- segmondy 1y ago
  why? it's basic, edit the config file. I just downloaded it, ~/.cache/share/crush/providers.json add your own or edit an existing one Edit api_endpoint, done.
- amdivia 1y ago
  FYI it works already even without this feature branch (you'll just have to add your provider and models manually) ``` { "providers": { "ollama": { "type": "openai", "base_url": "http://localhost:11434/v1", "api_key": "ollama", "models": [ { "id": "llama3.2:3b", "model": "Llama 3.2 3B", "context_window": 131072, "default_max_tokens": 4096, "cost_per_1m_in": 0, "cost_per_1m_out": 0 } ] } } } ```
navanchauhan 1y ago
sst/opencode
- metadat 1y ago
  But only a few models can actually execute commands effectively.. what is it, Claude and Gemini? Did I miss any?
  - rekram1-node 1y ago
    kiwi k2, Qwen3-Coder
    - metadat 1y ago
      Qwen3 issued such stupid commands I wouldn't count it in the same universe as Claude or Gem. I still need to test out Kimi, though- Cheers!
    - jmj 1y ago
      Seconding Kimi K2
    - tough 1y ago
      glm 4.5 air too
- beanjuiceII 1y ago
  has a ton of bugs
  - tough 1y ago
    works quite reliably with claude-code-router and any agentic SOTA model (not necessarily anthropic)
0x457 1y ago
Most of these agents work with any OpenAI compatible endpoints.
- oceanplexian 1y ago
  Actually not really. I spent at least an hour trying to get OpenCode to use a local model and then found a graveyard of PRs begging for Ollama support or even the ability to simply add an OpenAI endpoint in the GUI. I guess the maintainers simply don't care. Tried adding it to the backend config and it kept overwriting/deleting my config. Got frustrated and deleted it. Sorry but not sorry, I shouldn't need another cloud subscription to use your app. Claude code you can sort of get to work with a bunch of hacks, but it involves setting up a proxy and also isn't supported natively and the tool calling is somewhat messed up. Warp seemed promising, until I found out the founders would rather alienate their core demographic despite ~900 votes on the GH issue to allow local models https://github.com/warpdotdev/Warp/issues/4339 https://github.com/warpdotdev/Warp/issues/4339. So I deleted their crappy app, even Cursor provides some basic support for an OpenAI endpoint.
  - bachittle 1y ago
    LM Studio is probably better in this regard. I was able to get LM studio to work with Cursor, a product known to specifically avoid giving support to local models. The only requirement is if it uses servers as a middle-man, which is what Cursor does, you need to port forward.
    - tough 1y ago
      how did you do that? cursor seems very hard for me to modify to use local llms
  - steveharman 1y ago
    Just use Claude Router? Supports Ollama and most others. https://github.com/musistudio/claude-code-router https://github.com/musistudio/claude-code-router
  - 1y ago
elliotec 1y ago
Aider says they do, but I haven’t tried it. https://aider.chat/docs/llms.html https://aider.chat/docs/llms.html
- phaedrix 1y ago
  Aider has built in support for lm studio endpoints. https://aider.chat/docs/llms/lm-studio.html https://aider.chat/docs/llms/lm-studio.html
sharperguy 1y ago
What happens if you just point it at its own source and ask it to add the feature?
- segmondy 1y ago
  it will add the feature, I saw openAI make that claim that developers are adding their own features, saw Anthrophic make the same claim, and Aider's paul often says Aider wrote most of the code. I started building my own coding CLI for the fun of it, and then I thought, why not have it start developing features, and it does too. It's as good as the model. For ish and giggles, I just downloaded crush, pointed it to a local qwen3-30b-a3b which is a very small model and had it load the code, refactor itself and point bugs. I have never used LSP, and just wanted to see how it performs compared to treesitter.
jmj 1y ago
OpenHands let's you set any LLM you want. https://github.com/All-Hands-AI/OpenHands https://github.com/All-Hands-AI/OpenHands
segmondy 1y ago
all of them, you can even use claude-code with a local model

The big question - which one of these new agents can consume local models to a reasonable degree? I would like to ditch the dependency on external APIs - willing to trade some performance in lieu.

deleted 1y ago

[deleted]

jasonm23 1y ago

Crush has an open issue (2 weeks) to add Ollama support - it's in progress.

Aperocky 1y ago
nice, that would be my reason to use Crush.
- tempodox 1y ago
  Me too.
ggerganov 1y ago
They should add "custom endpoint" support instead [0]. [0] https://github.com/microsoft/vscode/issues/249605 https://github.com/microsoft/vscode/issues/249605
segmondy 1y ago
why? it's basic, edit the config file. I just downloaded it, ~/.cache/share/crush/providers.json add your own or edit an existing one Edit api_endpoint, done.
amdivia 1y ago
FYI it works already even without this feature branch (you'll just have to add your provider and models manually) ``` { "providers": { "ollama": { "type": "openai", "base_url": "http://localhost:11434/v1", "api_key": "ollama", "models": [ { "id": "llama3.2:3b", "model": "Llama 3.2 3B", "context_window": 131072, "default_max_tokens": 4096, "cost_per_1m_in": 0, "cost_per_1m_out": 0 } ] } } } ```

navanchauhan 1y ago

sst/opencode

metadat 1y ago
But only a few models can actually execute commands effectively.. what is it, Claude and Gemini? Did I miss any?
- rekram1-node 1y ago
  kiwi k2, Qwen3-Coder
  - metadat 1y ago
    Qwen3 issued such stupid commands I wouldn't count it in the same universe as Claude or Gem. I still need to test out Kimi, though- Cheers!
  - jmj 1y ago
    Seconding Kimi K2
  - tough 1y ago
    glm 4.5 air too
beanjuiceII 1y ago
has a ton of bugs
- tough 1y ago
  works quite reliably with claude-code-router and any agentic SOTA model (not necessarily anthropic)

0x457 1y ago

Most of these agents work with any OpenAI compatible endpoints.

oceanplexian 1y ago
Actually not really. I spent at least an hour trying to get OpenCode to use a local model and then found a graveyard of PRs begging for Ollama support or even the ability to simply add an OpenAI endpoint in the GUI. I guess the maintainers simply don't care. Tried adding it to the backend config and it kept overwriting/deleting my config. Got frustrated and deleted it. Sorry but not sorry, I shouldn't need another cloud subscription to use your app. Claude code you can sort of get to work with a bunch of hacks, but it involves setting up a proxy and also isn't supported natively and the tool calling is somewhat messed up. Warp seemed promising, until I found out the founders would rather alienate their core demographic despite ~900 votes on the GH issue to allow local models https://github.com/warpdotdev/Warp/issues/4339 https://github.com/warpdotdev/Warp/issues/4339. So I deleted their crappy app, even Cursor provides some basic support for an OpenAI endpoint.
- bachittle 1y ago
  LM Studio is probably better in this regard. I was able to get LM studio to work with Cursor, a product known to specifically avoid giving support to local models. The only requirement is if it uses servers as a middle-man, which is what Cursor does, you need to port forward.
  - tough 1y ago
    how did you do that? cursor seems very hard for me to modify to use local llms
- steveharman 1y ago
  Just use Claude Router? Supports Ollama and most others. https://github.com/musistudio/claude-code-router https://github.com/musistudio/claude-code-router
- 1y ago

elliotec 1y ago

Aider says they do, but I haven’t tried it. https://aider.chat/docs/llms.html https://aider.chat/docs/llms.html

phaedrix 1y ago
Aider has built in support for lm studio endpoints. https://aider.chat/docs/llms/lm-studio.html https://aider.chat/docs/llms/lm-studio.html

sharperguy 1y ago

What happens if you just point it at its own source and ask it to add the feature?

segmondy 1y ago
it will add the feature, I saw openAI make that claim that developers are adding their own features, saw Anthrophic make the same claim, and Aider's paul often says Aider wrote most of the code. I started building my own coding CLI for the fun of it, and then I thought, why not have it start developing features, and it does too. It's as good as the model. For ish and giggles, I just downloaded crush, pointed it to a local qwen3-30b-a3b which is a very small model and had it load the code, refactor itself and point bugs. I have never used LSP, and just wanted to see how it performs compared to treesitter.

jmj 1y ago

OpenHands let's you set any LLM you want. https://github.com/All-Hands-AI/OpenHands https://github.com/All-Hands-AI/OpenHands

segmondy 1y ago

all of them, you can even use claude-code with a local model