6 ms·
DeepSeek v4.1 Flash Is Now Our Best Hacking Model
- fwip 18d agoEdit: Comment deleted - no longer useful.
- Aldipower 18d agoYou can have 100 runs for the price of the Claude Max plan?
- trollbridge 18d agoOr the price of a GPT Pro 20X plan... er, wait, except you can't go sign up for one of those right now.
- tyingq 18d agoApparently they updated it perhaps based on your comment? > . The accepted runs cost $4.65. Failed attempts and replacement runs increased the complete cost to $5.14.
- fwip 18d agoOh - I see that now. It's possible I missed it originally - I did read the article but I was skimming quickly. Mea culpa, if so.
- TuxSH 18d agoI find this - or perhaps the title - a bit surprising. I've benchmarked GLM 5.3 and DSv4.1-F on my fully-annotated decomp of the Nintendo 3DS's kernel, which I have a good mental understanding of, tasking them to find vulns and other bugs (in Max mode w/ subagents). GLM 5.3 founds almost all the vulns in 30min for $22, while DS only found one vuln for $2 in 40min. Perhaps DS works better where targets have low-hanging fruits than can be found fast?
- simlevesque 18d agoIn your example, couldn't you parallelize DS's work more ? You could have 11 times as many agents for the same price.
- d5lt5 18d agoDS has perf issues if you parallelize it heavily (24+), especially when the context window is above the limit, on a single machine (with custom llm gateway): it fails 4x more often, and is 2x slower than gpt-5.6.
- TuxSH 18d agoBoth DS and GLM had the same numbers of subagents, 5 or so. But, well, number of subagents doesn't make a difference if model is dumb (GPT 5.4 High, in May,in Chat mode outperformed what I see with DS4.1-F). That being said, pricing model makes a huge difference for "find at least one" tasks: with API/PAYG if you have a chance to save 90%, you go for it, whereas with subscriptions it is optimal to burn all your remaining allowance right before reset
- mariopt 18d agoMakes sense, GLM is a lot better than DS v4.1, I found the same results in other domains. Given how fast and cheap DS is, it's just an ideal model with enough "IQ" to let it loose. Another thing they left out of the article, DS becomes really good with if provide custom tools for the task, on it's own it's mediocre.
- nickysielicki 18d agoWhen the history books are written and all is said and done, the hubris of this moment where all the American labs decided to punk their investors and join hand in hand in agreeing to let the Chinese win forever is going to be the main story.
- EGreg 18d agoWin what? The race to the bottom always has this competitive language. “If we ban CFCs now the Chinese will win!” “If we ban chemical weapons, nuclear weapons, etc etc our enemies will triumph! They won’t stop!” “If we switch to biodegradeable plastic then our rivals will have an advantage.” “If we dont externalize the costs to our population, then they will, and then will win!” I think workflows can do the job agents do, 20x cheaper and more predictably and safely. They can completely displace agents, just as HFCs displaced CFCs and then we were able to ban CFCs and phase them out through international COOPERATION. The language of COOPERATION is what saves us vs COMPETITION is all about cutting corners and externalizing costs. Google the Montreal Protocol, Geneva Conventions, Nuclear Non Proliferation Treaty, Unleaded Gasoline etc etc. Agents have got to be marginalized. They are just popular because the labs need to make a ton of money for their investors and recoup their massive spending on training models.
- airstrike 18d agoThose comparisons are pretty irrelevant You can't compare banning football to banning genetic experiments and say "they are both bans and therefore directly comparable"
- EGreg 18d agoNo one was talking about football. We're talking about replacing most uses of Agents with uses of Workflows. That's the key first step, without which the industry will cry "don't regulate us... do you want China to WIN???"
- palmotea 18d ago> When the history books are written and all is said and done, the hubris of this moment where all the American labs decided to punk their investors and join hand in hand in agreeing to let the Chinese win forever is going to be the main story. Come on. Lecturing about hubris when your message is damn the consequences, full speed ahead? If it's a race to build the torment nexus, or a race with a nonzero chance of building the torment nexus by accident, I don't care about winning.
- jrflo 18d agoSeems pretty bold to claim deepseek is the "best hacking model" while providing zero comparisons to other models...
- sva_ 18d agoNotice the qualifier "our", that is the one they have access to.
- dude250711 18d agoWhat if there would be a separate category for distilled models?
- wg0 18d agoDeepSeek is underrated. Basically all Chinese models are good enough for day to day coding at this point. The 2 trillion dollar ROI on anthropic alone? Good luck with that.
- Art9681 18d agoGood luck building something of value with "day to day coding". If you're going to compete in the future economy, you're going to compete at the frontier. The low hanging fruit will be of no value. If I can recreate your SaaS with an open weight model at a fraction of the cost, then your SaaS has no value.
- blck-b 18d ago[dead]
- cl-coder 18d agoIf I can recreate your "frontier" SaaS with day-to-day coding, it has no value. Or perhaps the few months' lead was never the deciding factor.
- serf 18d agothe vast majority of successful SaaS offer capabilities or services no where near the frontier. furthermore i'd argue that the success of an saas, or really any startup, has more to do with things aside from the code and service itself. Finance management, social connections and effects, word of mouth, networking -- probably all more important than whether or not the todo app uses a functional language and is formally proven.
- nvme0n1p1 18d agoWhat the hell are you yammering about. Everything in the economy exists at different quality/price tradeoffs, everything from tomato sauce at the grocery store to landscaping services to SaaS. "Compete at the frontier" is some maximalist buzzword nonsense. Do you think Ikea is crying because their furniture is cheap and they're not "competing at the frontier"?
- 18d ago
- Art9681 18d agoAs opposed to what? Is enclave.ai signed up for GPT Cyber or Glasswing?
- habosa 18d agoDeepSeek models have such good benchmark performance, amazing pricing, and the team over there seems to be widely considered impressive. I just haven't found them to be very good? I've had a ton more success with the GLM models (since 5.2 anyway). Maybe I'm just holding it wrong, DS models seem to get stuck in loops or tell me nonsense. GLM feels like budget Claude.
- tired_star_nrg 18d agoIt’s funny, in oh-my-pi I’ve had much better luck with DS 4.1 Flash than GLM 5.3 flash. Glm seems to just constantly reason in circles before attempting a tool call
- tacomagick 18d agoDepends on a lot of factors, AFAIK Claude Code (TUI Harness) does not work well with Deepseek, but Opencode is surprisingly good.
- gpugreg 18d ago> DS models seem to get stuck in loops I have never had looping issues with DeepSeek models. Which provider/serving framework and harness are you using?
- pimeys 18d agoMy experience is complete opposite from yours. The V4 Flash was already quite good, but V4.1 is really very close to SOTA in my books. I've eval'd these models for weeks against Gemini 3.8, Kimi K3 and Opus 5, and DeepSeek absolutely wins these evals. It's really crazy. We price per token our customers. If Kimi was about 30% of the price of Opus 5 for the same quality, DeepSeek is 1/10th of a price of Kimi K3. We've come down in price so much that I seriously cannot recommend other models before they reduce their pricing. And what is really interesting is its programming ability. As I've said in my previous comments, I use agents a lot in my work. Since early Opus days until now I have 7-8 agents working in parallel for different tasks. Rust, design, GEPA, evals, analysis. For a long time Kimi K3 was the best model for this work, and before that GPT 5.5. But I still can't really believe how well DeepSeek works here. I really try to find faults from it, trying to see that it must be doing sloppy work and be worse than the others. But it does not. It finishes every task I give to it. And the cost per task is under a dollar, usually 15-30 cents. In comparison the same task with Kimi would be 3-15 dollars; sometimes closing to 100. And before that with GPT 5.5 a 800 dollar task was not uncommon if I spent days evaluating models. Now it's less than a dollar. For me if the other providers will not drop their prices dramatically in the coming weeks I see no reason to use them. Even with a 200 dollar subscription, paying per token for DeepSeek is better value. My harness: https://omp.sh/ https://omp.sh/
- strictnein 18d ago[dead]
- fra 18d agoInteresting result, but the writing is very poor.
- gertlabs 18d agoWe ran v4.1 Flash through our evaluations and found it to be smarter and faster than V4 Flash, with a commensurate price bump. Some notes: - Because it's a heavy reasoner, it sits near Gemini 3.7 Flash on the Pareto front (not as cheap as the price suggests in practice). - Closer than expected to the top open weights models (GLM 5.3 and Kimi K3) in agentic coding, at lower cost. - Chinese models have always been strong iterators in an agentic harness. This model is no different, reaching an average percentile ~20% higher when given a harness vs a one-shot solution. That one-shot fluid intelligence is what makes a model feel smart, though, and typically results in fewer attempts/tokens to solve a problem, and American frontier models are still far ahead in that department. The new architecture is interesting. It puts pricing between their old Flash and Pro lineups, suggesting they might be abandoning their super-cheap flash models (which weren't that fast due to heavy reasoning) and their pro models (which sort of flopped and weren't consistently better than their flash models, despite the size/cost) and shipping a strong intermediate that competes with the Gemini Flash series. Data at https://gertlabs.com/rankings https://gertlabs.com/rankings
- deleted 18d ago[deleted]
- pelzatessa 18d agoHow do I make deepseek "hack" my source code? do I just start my coding agent in my directory and command it to "find vulnerabilities", or is there some more sophisticated software to do that?
- AshamedBadger56 18d agoStart with an existing open source AI pentesting / source code scanning tool. Shannon and Strix are 2 that come to mind. They provide the tools, prompt, and frameworks for the model to use. https://github.com/KeygraphHQ/shannon https://github.com/KeygraphHQ/shannon https://github.com/usestrix/strix https://github.com/usestrix/strix
- hactually 18d agowe recently got this running in 192gb of vRAM and using it with the Klaudia harness has been incredible for driving out work that we'd need to use Opus and Fable for previously
- Grimblewald 18d agoI'm a deepseek fanboy, but has anyone else found flash to not meet expectations? I've found it to be wildly bad at doing as asked, overengineering, and always assuming instead of reading even if told to read things in full before doing anything. It makes wildly silly mistakes in code and so far has been quite frustrating to work with. Maybe its just what i work on that its particularly bad at but in general feels like a strict regression on previous offerings.
- kelvintestsai 17d ago[dead]
- anotherCodder 17d ago[dead]