2 ms·
> if someone would kindly quantise this down for the 64GB paupers The Q4_K_M is 75GB. I'm exactly at 64GB, and I wouldn't quantize it further. Instead, do part
by Archit3ch 3mo ago
> if someone would kindly quantise this down for the 64GB paupers
The Q4_K_M is 75GB. I'm exactly at 64GB, and I wouldn't quantize it further. Instead, do partial weight residency and stream the rest from SSD.
- bitexploder 2mo agoI am going to see what I can get out of this streaming from SSD vs. Qwen 27B. Laguna is a seriously impressive model. I already have a lot of success with 27B A3B.