4 ms·
Confirmed. Currently running Mixtral 8x7B gguf (Q8_0) on a Macbook Pro M1 Max w 64GB ram, and RAM usage is sitting at 48.8 GB.
by discordance 3y ago
Confirmed. Currently running Mixtral 8x7B gguf (Q8_0) on a Macbook Pro M1 Max w 64GB ram, and RAM usage is sitting at 48.8 GB.
- karolist 3y agoHow many t/s?
- discordance 3y agoAround 15 - 20 t/s
- karolist 3y agoThank you, got the same build M1 Max during Christmas B&H sale and can confirm it's amazing for running local LLMs.