3 ms·
This is a good guide. Ollama and Llamafile are two of my top choices for this. On macOS it's worth investigating the MLX ecosystem. The easiest way to do that
by simonw 2y ago
This is a good guide. Ollama and Llamafile are two of my top choices for this.
On macOS it's worth investigating the MLX ecosystem. The easiest way to do that right now is using LM Studio (free but proprietary), or you can run the MLX libraries directly in Python. I have a plugin for my LLM CLI tool that uses MLX here: https://simonwillison.net/2025/Feb/15/llm-mlx/ https://simonwillison.net/2025/Feb/15/llm-mlx/
- archerx 2y ago+1 for Llamafile. They just released a new release and it supports Deepseek R1 models. I have a blank Llamafile and load models into it using a .bat file that I edit as needed. I have a custom webui that I made that lets me save and organize my sessions. Whisperfile is also amazing. I used it to transcribe a podcast I hosted 20 years ago and it was fast and the errors it made were acceptable.
- iugtmkbdfil834 2y agoAgreed on Ollama ( especially as a starting point since it makes a lot of the initial setup super easy for anyone new ). OTOH, you may quickly become aware of the limitations imposed with its defaults ( for example, low context -- to be fair, it can be adjusted ).