3 ms·
Yeah, that's my experience. It's a big "wow" factor to get a non-trivial LLM running on my Mac, but it's actually not that useful. Like trying to use Photosho
by petcat 2mo ago
Yeah, that's my experience. It's a big "wow" factor to get a non-trivial LLM running on my Mac, but it's actually not that useful. Like trying to use Photoshop at 8 FPS.
- coldtea 2mo agoRegarding this analogy, fps don't matter as much for Photoshop, since it's not an immediate mode GUI. 8 fps would be quite ok for comfortably getting feedback on live image filters and such.
- LeBit 2mo agoIt’s not because you didn’t find use cases for local LLMs that there are none. I use local LLMs on my Mac Mini M4 Pro with 48G to review text messages tone, act as a text correction tool, act as a code review tool, to do code agent work, generate code snippets, etc Gemma 4 26B A4B gives me steady 20 tps.