4 ms·

Curious how people feel about this compared to DS4 Flash, given they are pretty close in size. Also curious how well it holds up to heavy quantization. DS4 Fla

Curious how people feel about this compared to DS4 Flash, given they are pretty close in size. Also curious how well it holds up to heavy quantization. DS4 Flash can currently run reasonably well on systems with ~96gb+ RAM, I wonder if Hy3 can compete there.

nunodonato 3mo ago
DS4-Flash is not only "significantly" smaller, it will also benefit from a lot more speed thanks to DSpark
- Catloafdev 3mo ago
  299B for Hy3 vs 284B* for Flash Edit: fixed, got bad info
  - nshotton 3mo ago
    flash is 284b isnt it? https://artificialanalysis.ai/models/deepseek-v4-flash https://artificialanalysis.ai/models/deepseek-v4-flash
    - Catloafdev 3mo ago
      Oh, it is. I was looking at the Huggingface repo which listed the lower number at the top of the page, looks like that's wrong.
      - nunodonato 3mo ago
        Same, I was relying on that. Weird!
wolttam 3mo ago
Hy3 lacks the DSv4 architecture's KV Cache efficiency. Whereas I can run DSv4 Flash on a pair of DGX Sparks and have enough memory left over for 3M tokens of KV cache, with Hy3 (quantized to FP4), there is only room for ~130K tokens of KV cache.
- ignoramous 3mo ago
  Lower context window notwithstanding, Hy3's coding benchmarks hold their own against DeepSeek v4 Pro & MiMo v2.5 Pro. That's quite something for a model priced like DeepSeek v4 Flash & MiMo v2.5 (for non-cached tokens), which are 3x cheaper than their respective Pro variants.
  - wolttam 3mo ago
    It's impressive indeed. I would also expect the next checkpoint of DSv4 Flash to come in somewhere at this level (DeepSeek has had over 2 months to continue training since it released). It's exciting that the open models continue to get better and more efficient across the board!
UncleOxidant 3mo ago
That's a 2-bit quant of DS4 flash. You're probably better off running Qwen3.6-27B at Q8.
- sosodev 3mo ago
  I suspect it would depend on the task. DS4-flash does, as previously mentioned, handle quantization very well. Even at 2-bit it's still very coherent.
- spmurrayzzz 3mo ago
  I think its good advice to test both on your own evals for sure, but the MoE parameters are already natively FP4 in ds4. Dropping to 2bpw isn't as big of a loss as it seems (and as corroborated by antirez's work). Its also only 13B active, so your decode speed would be nearly 2x that of Qwen3.6-27B. So there are other latent benefits as well.
  - verdverm 3mo ago
    z-lab has been dropping dflash addons for a lot of models https://huggingface.co/collections/z-lab/dflash https://huggingface.co/collections/z-lab/dflash I'm running the qwen3.6-27B + dflash on a spark and tgen is way up, but keep the draft count low, acceptance rate is terrible beyond half a dozen and it requires more memory
    - bjconlan 3mo ago
      Wow thanks for pointing this out! This is actually what I was hoping would happen when the deepspec stuff dropped! And having zlab create these confirms my bias that I think open models are the way.
- Catloafdev 3mo ago
  For most coding or agentic tasks, Qwen 3.6 27B likely outperforms, yes. For 'general intelligence', DS4 Flash seems to be a noticeable step up still.
- ckocagil 3mo ago
tarruda 3mo ago
> given they are pretty close in size One thing that might not be obvious about about DSV4 is how much innovation the Deepseek team implemented in its architecture. When llama.cpp fully supports its lightning indexer, the full 1M context will only require about 6G of RAM. So even though they are similar in size, I believe Deepseek will be much more efficient in that regard. > I wonder if Hy3 can compete there Highly depends on how well Hy3 is resilient to quantization. DSV4 is useful even at 2-bit quants.
- throwa356262 3mo ago
  I have been telling people exactly this the last few months. We have not seen the full power of deepseek v4 yet.
Computer0 3mo ago
I don’t like DS4 in my experiences with it I still prefer qwen locally and glm on api
nmfisher 3mo ago
I was playing with Hy3 via openrouter yesterday (and I've also been using DS4 Flash/Pro as a daily driver since I cancelled my Anthropic sub a week ago). I've found DS4 Flash to be very temperental (via Claude Code). The speed is great, but it often builds a completely wrong mental model and charges off down the wrong path. I find myself needing to rein it in regularly (and also compact the history, which undercuts the whole cache price advantage). Hy3 isn't as fast, but so far it seems to stay on track much more reliably than DS4 Flash. It also doesn't seem to degrade as much with longer context. I'm not sure what the real pricing is, but I feel like it's a very competitive model. As an aside, I also nabbed a 50m token pack for LongCat 2.0 to give it a whirl. Not free, but it's so cheap they're basically giving it away. Very impressed too - seems roughly on par with Hy3. Not frontier-level intelligence, but a dependable workhorse that can navigate a codebase well and can reliably execute what you tell it to do.

nunodonato 3mo ago

DS4-Flash is not only "significantly" smaller, it will also benefit from a lot more speed thanks to DSpark

Catloafdev 3mo ago
299B for Hy3 vs 284B* for Flash Edit: fixed, got bad info
- nshotton 3mo ago
  flash is 284b isnt it? https://artificialanalysis.ai/models/deepseek-v4-flash https://artificialanalysis.ai/models/deepseek-v4-flash
  - Catloafdev 3mo ago
    Oh, it is. I was looking at the Huggingface repo which listed the lower number at the top of the page, looks like that's wrong.
    - nunodonato 3mo ago
      Same, I was relying on that. Weird!

wolttam 3mo ago

Hy3 lacks the DSv4 architecture's KV Cache efficiency. Whereas I can run DSv4 Flash on a pair of DGX Sparks and have enough memory left over for 3M tokens of KV cache, with Hy3 (quantized to FP4), there is only room for ~130K tokens of KV cache.

ignoramous 3mo ago
Lower context window notwithstanding, Hy3's coding benchmarks hold their own against DeepSeek v4 Pro & MiMo v2.5 Pro. That's quite something for a model priced like DeepSeek v4 Flash & MiMo v2.5 (for non-cached tokens), which are 3x cheaper than their respective Pro variants.
- wolttam 3mo ago
  It's impressive indeed. I would also expect the next checkpoint of DSv4 Flash to come in somewhere at this level (DeepSeek has had over 2 months to continue training since it released). It's exciting that the open models continue to get better and more efficient across the board!

UncleOxidant 3mo ago

That's a 2-bit quant of DS4 flash. You're probably better off running Qwen3.6-27B at Q8.

sosodev 3mo ago
I suspect it would depend on the task. DS4-flash does, as previously mentioned, handle quantization very well. Even at 2-bit it's still very coherent.
spmurrayzzz 3mo ago
I think its good advice to test both on your own evals for sure, but the MoE parameters are already natively FP4 in ds4. Dropping to 2bpw isn't as big of a loss as it seems (and as corroborated by antirez's work). Its also only 13B active, so your decode speed would be nearly 2x that of Qwen3.6-27B. So there are other latent benefits as well.
- verdverm 3mo ago
  z-lab has been dropping dflash addons for a lot of models https://huggingface.co/collections/z-lab/dflash https://huggingface.co/collections/z-lab/dflash I'm running the qwen3.6-27B + dflash on a spark and tgen is way up, but keep the draft count low, acceptance rate is terrible beyond half a dozen and it requires more memory
  - bjconlan 3mo ago
    Wow thanks for pointing this out! This is actually what I was hoping would happen when the deepspec stuff dropped! And having zlab create these confirms my bias that I think open models are the way.
Catloafdev 3mo ago
For most coding or agentic tasks, Qwen 3.6 27B likely outperforms, yes. For 'general intelligence', DS4 Flash seems to be a noticeable step up still.
ckocagil 3mo ago

tarruda 3mo ago

> given they are pretty close in size One thing that might not be obvious about about DSV4 is how much innovation the Deepseek team implemented in its architecture. When llama.cpp fully supports its lightning indexer, the full 1M context will only require about 6G of RAM. So even though they are similar in size, I believe Deepseek will be much more efficient in that regard. > I wonder if Hy3 can compete there Highly depends on how well Hy3 is resilient to quantization. DSV4 is useful even at 2-bit quants.

throwa356262 3mo ago
I have been telling people exactly this the last few months. We have not seen the full power of deepseek v4 yet.

Computer0 3mo ago

I don’t like DS4 in my experiences with it I still prefer qwen locally and glm on api

nmfisher 3mo ago

I was playing with Hy3 via openrouter yesterday (and I've also been using DS4 Flash/Pro as a daily driver since I cancelled my Anthropic sub a week ago). I've found DS4 Flash to be very temperental (via Claude Code). The speed is great, but it often builds a completely wrong mental model and charges off down the wrong path. I find myself needing to rein it in regularly (and also compact the history, which undercuts the whole cache price advantage). Hy3 isn't as fast, but so far it seems to stay on track much more reliably than DS4 Flash. It also doesn't seem to degrade as much with longer context. I'm not sure what the real pricing is, but I feel like it's a very competitive model. As an aside, I also nabbed a 50m token pack for LongCat 2.0 to give it a whirl. Not free, but it's so cheap they're basically giving it away. Very impressed too - seems roughly on par with Hy3. Not frontier-level intelligence, but a dependable workhorse that can navigate a codebase well and can reliably execute what you tell it to do.