2 ms·
Isn't Deepseek V4 Flash still like 150+ GB even at Q4?
by johncalvinyoung 3mo ago
Isn't Deepseek V4 Flash still like 150+ GB even at Q4?
- kamranjon 3mo agoI think most people running deepseek locally are using DS4: https://github.com/antirez/ds4 https://github.com/antirez/ds4 Which provides a 2-bit quant and a mixed 2-bit/4-bit quant - which range from 80gb to 97gb Deepseek is particularly well suited for quantization and the quality of the 2bit quants antirez has provided have been validated by folks like ggernov of llama.cpp project