3 ms·

I'm looking for a setup for local development with local qwen on my macbook. I tried localforge with mlx_lm.server but it failed to communicate (I saw a proof o

I'm looking for a setup for local development with local qwen on my macbook. I tried localforge with mlx_lm.server but it failed to communicate (I saw a proof of concept on their page but now it seems to fail on "empty response" which in reality is not empty) Anyone could recommend a solution?

zhobbs 1y ago
Possible I'm misunderstanding what you're trying to do, but ollama works well for me for local inference with qwen on my Macbook Pro (32GB).
- nateb2022 1y ago
  Yup, also using Ollama and on a Macbook Pro. Ollama is #1
  - p0w3n3d 1y ago
    But isn't ollama only local chat? Or I am missing something? I'd like to setup it as a server for my usages on another laptop (use it as my local AI hub) and would love to integrate it with some IDE using MCP
    - grosswait 1y ago
      No, it can listen on 0.0.0.0 or you can serve it through a proxy