3 ms·
What model sizes do you run locally? Anything that would work on a 16GB M1?
by 404mm 2y ago
What model sizes do you run locally? Anything that would work on a 16GB M1?
- griomnib 2y agoI have an A6000 with 48GB VRAM I run from a local server and I connect to it using Enchanted on my Mac.
- mark_l_watson 2y agoI ha e a 32G M2, but most local models I use fit into my 8G old M1 laptop. I can run the QwQ 32G model with Q4 on my 32G M2. I suggest using https://Ollama.com https://Ollama.com on Mac, Windows, and Linux. I experiments with all options on Apple Silicon and liked Ollama best.