3 ms·
I was so excited that it is open source until i realised the model required 1.5To of VRAM. Unsloth has compressed it in 1bit at about 570gb VRAM with 75% accura
by fumber 3mo ago
I was so excited that it is open source until i realised the model required 1.5To of VRAM. Unsloth has compressed it in 1bit at about 570gb VRAM with 75% accuracy, that's almost mac studio territory...
- nateb2022 3mo agoYou can run DeepSeek v4 Flash at 4bit in ~150 GB VRAM, and it would absolutely destroy a 1 bit quant of K3.
- AmazingTurtle 3mo ago75% quality... it gets worse the longer the context.