3 ms·
Yeah it 100% depends what you want the model to do. Some tasks, like extraction, summarization, or simple tool calling (e.g. "turn on my desk lamp") are very do
by aftbit 4mo ago
Yeah it 100% depends what you want the model to do. Some tasks, like extraction, summarization, or simple tool calling (e.g. "turn on my desk lamp") are very doable with tiny models. Others, like coding or more advanced agentic workflows can demand much more powerful models. I was thinking from the lens of coding or running _big_ data extraction pipelines (think ~8 billion pages).
- EagnaIonat 4mo ago> thers, like coding or more advanced agentic workflows can demand much more powerful models. You can do coding and agentic fine. For coding I use qwen3.6:35b-mlx and agentic granite4.1:3b works fine. These are the models I use. - granite4.1:3b - granite4.1:30b - gpt-oss:20b - gpt-oss:120b (less so now) - mistral-small3.2 - qwen3.6:35b-mlx There will always be use cases that don't sit on your laptop, but most of what can be done can be done locally, it just requires a good framework to sit on it.
- packetlost 4mo agoWhy do you like gpt-oss-120b less now? What replaced it?
- aftbit 4mo agoIt's very likely to hallucinate. I'm mostly using Gemma 4 31B now when I need something offline. It is a very strong model for its size.
- EagnaIonat 4mo agoFor me the other models are faster and results are good enough.