3 ms·
Running DeepSeek V4 without extreme quantization locally requires a lot of hardware. The IQ2 quants that fit into 128GB machines are very degraded.
by Aurornis 6mo ago
Running DeepSeek V4 without extreme quantization locally requires a lot of hardware.
The IQ2 quants that fit into 128GB machines are very degraded.
- binyu 6mo agoThat is true, it is a 1.6T parameters model so it requires a great deal of memory. I also heard there's a 2bit quantization that works well on Apple metal.