3 ms·
> The fact a laptop can run 70B+ parameter models is a miracle Most laptops with 64+GB of RAM can run a 70B model at 4-bit quantization. It’s not a miracle, it
by dragonwriter 3y ago
> The fact a laptop can run 70B+ parameter models is a miracle
Most laptops with 64+GB of RAM can run a 70B model at 4-bit quantization. It’s not a miracle, it’s just math. M2 can do it faster than systems with slower memory bandwidth.
- deleted 3y ago[deleted]