4 ms·

Here's a 4bit 70B parameter model, https://www.youtube.com/watch?v=5ktS0aG3SMc https://www.youtube.com/watch?v=5ktS0aG3SMc (deepseek-r1:70b Q4_K_M) on a M4 Max

by phonon 1y ago

Here's a 4bit 70B parameter model, https://www.youtube.com/watch?v=5ktS0aG3SMc https://www.youtube.com/watch?v=5ktS0aG3SMc (deepseek-r1:70b Q4_K_M) on a M4 Max 128 GB. Usable, but not very performant.