4 ms·

Is there a way to achieve the same with ollama?

• simonw 1y ago
  Yes, Ollama has Qwen 3 and it works great on a Mac. It may be slightly slower than MLX since Ollama hasn't integrated that (Apple Silicon optimized) library yet, but Ollama models still use the Mac's GPU. https://ollama.com/library/qwen3 https://ollama.com/library/qwen3
  • avetiszakharyan 1y ago
    Yes, i did that but its not apple silicon optimized so it was taking forever for 30b models. So its ok, but its not fantastic
    • spmurrayzzz 1y ago
      You can just use llama.cpp instead (which is what ollama is using under the hood via bindings). Just need to make sure youre using commit `d3bd719` or newer. I normally use this with nvidia/cuda, but tested on my mbp and havent had any speed issues thus far. Alternatively, LMStudio has MLX support you can use as well.