2 ms·
After using a highly capable 2-bit quant as my daily driver for months now, I get pretty excited about releases like this. After a few days for the kinks to be
by kamranjon 3mo ago
After using a highly capable 2-bit quant as my daily driver for months now, I get pretty excited about releases like this. After a few days for the kinks to be worked out, I’ll be excited to try it.
- digdugdirk 3mo agoWhat model? And what hardware do you run it on? I find these style of models are great, but fail hard, and fail randomly. I'd be hesitant to use it for a daily driver, but I'm using dual 3060s, so it's not like I'm quantizing a frontier model here. How do you find the overall experience? And do you have any special sauce or recommendations for going this route?
- kamranjon 3mo agoI’m using DeepSeek V4 Flash on 128gb mbp - it’s a bit different using a 200b+ param model. It’s MoE so performance is acceptable. It will still malform a tool call every now and then, but the capabilities are so far ahead anything else that the majority of the time it works really well and solves really complex problems.
- apitman 3mo agoI've got dual 3060s as well. What's the best models you've found for this setup?
- RALaBarge 3mo agoWhat have you been using?
- kamranjon 3mo agoDeepSeek V4 Flash with DwarfStar: https://github.com/antirez/ds4 https://github.com/antirez/ds4 The 2 bit quants are really good. I have a lot of memory so I can squeeze it all in at ~80gb.
- pylotlight 3mo agoBut isn't it running basically 1 request at a time? This would make agentic coding difficult right? Compared to running as many sequential tests as you want via api?
- kamranjon 3mo agoYea I don’t use sub-agent style workflows. I often use planning patterns and generate markdown for really complex tasks, but never got into the sub-agent thing. I am likely closer to AI-assisted, though I am heavily using pi coding agent and my editor is basically just for viewing files and changes now.