3 ms·

This is very convenient and nice! But I could not get it to work with the best small models available for Ollama for programming, like https://ollama.com/MFDoo

This is very convenient and nice! But I could not get it to work with the best small models available for Ollama for programming, like https://ollama.com/MFDoom/deepseek-coder-v2-tool-calling https://ollama.com/MFDoom/deepseek-coder-v2-tool-calling for example.

smcleod 1y ago
That's a really old model now. Even the old Qwen 2.5 coder 32b model is better than DSv2
- codingmoh 1y ago
  I want to add support for qwen 2.5 next
  - manmal 1y ago
    QwQ-32 might be worth looking into also, as a high level planning tool.
    - codingmoh 1y ago
      Thank you so much!
  - smcleod 1y ago
    Hopefully Qwen 3 and maybe if we're lucky Qwen 3 Coder might be out this week too.
    - smcleod 1y ago
      Also GLM 4 is pretty amazing - https://www.reddit.com/r/LocalLLaMA/comments/1k4w9p2/i_uploaded_glm432b0414_glmz132b0414_q4_k_m_to/ https://www.reddit.com/r/LocalLLaMA/comments/1k4w9p2/i_uploa...
      - codingmoh 1y ago
        Thanks, I'll have a look
codingmoh 1y ago
Thanks so much! Was the model too big to run locally? That’s one of the reasons I went with phi-4-mini - surprisingly high quality for its size and speed. It handled multi-step reasoning, math, structured data extraction, and code pretty well, all on modest hardware. Phi-1.5 / Phi-2 (quantized versions) also run on raspberry pi as others have demonstrated.
- xyproto 1y ago
  The models work fine with "ollama run" locally. When trying out "phi4" locally with: open-codex --provider ollama --full-auto --project-doc README.md --model phi4:latest I get this error: OpenAI rejected the request. Error details: Status: 400, Code: unknown, Type: api_error, Message: 400 registry.ollama.ai/library/phi4:latest does not support tools. Please verify your settings and try again.