3 ms·
DeepSeek V4 with 1 million token context window is pretty powerful, although still not there. There's hope that Opus 4.5 level performance locally is not that f
by binyu 5mo ago
DeepSeek V4 with 1 million token context window is pretty powerful, although still not there. There's hope that Opus 4.5 level performance locally is not that far away.
- tuananh 5mo agoFrom what I read, ds v4 is very close with opus 4.6 performance.
- DeathArrow 5mo agoThe full model is, not the quantized versions.
- tuananh 5mo agoyeah that goes without saying. how can openweight, quantized version beat SOTA :)
- array_key_first 5mo agoWell it depends on the task. For agentic coding, more is more, but for tasks that normal consumers use them for there really is a ceiling. OCR, text to speech, that type of thing doesn't really improve when going to a SOTA model, so you'd just be wasting your money. I think local LLMs have more value than software engineers give them credit for.
- tuananh 5mo agototally agree with that. local llm doesn't need to match SOTA performance in order to be useful.
- Aurornis 5mo agoRunning DeepSeek V4 without extreme quantization locally requires a lot of hardware. The IQ2 quants that fit into 128GB machines are very degraded.
- binyu 5mo agoThat is true, it is a 1.6T parameters model so it requires a great deal of memory. I also heard there's a 2bit quantization that works well on Apple metal.