8 ms·
Anthropic's open-source framework for AI-powered vulnerability discovery
- bigmattystyles 4mo agoI wonder how this sort of product is going over at Coverity and others like it. Proper SAST vendors I mean. Is it an existential threat?
- rms2ds 4mo agoIf I had to guess, they'l eventually just add it into their own product and hike the prices up to cover tokens lol.
- lanyard-textile 4mo ago>This repo is not maintained and is not accepting contributions. Hm :)
- spacebacon 4mo agoThis one is and should be adapted to every frozen model ASAP. https://github.com/space-bacon/SRT https://github.com/space-bacon/SRT Significantly improve every frozen model overnight. LFG.
- Hamuko 4mo agoWhy isn't Claude maintaining it?
- skeledrew 4mo agoThey pretty much saying the efficacy of the tool can be tested by anyone to determine if it's worth purchasing the more polished and up-to-date commercial offering.
- politelemon 4mo agoThey must have solved coding that well.
- trilogic 4mo agohttps://github.com/Mainframework/Anthropic-Cybersecurity-Skills https://github.com/Mainframework/Anthropic-Cybersecurity-Ski... Be aware: the .py/s will not pass the antivirus but basically they do the job.
- simonw 4mo agoI wonder how much this thing costs to run. https://github.com/anthropics/defending-code-reference-harness/blob/main/docs/troubleshooting.md#rate-limits https://github.com/anthropics/defending-code-reference-harne... says: > As a rough guideline, expect ~10K uncached input tokens/min and ~2K output tokens/min per agent. You can scale parallelism up to your account's ITPM limit (roughly 10 agents per 100K ITPM). My guess would be hundreds of dollars with Opus and thousands of dollars with Mythos.
- Analemma_ 4mo agoI mean, you don't need to run it all the time, right? You do it once over your entire existing codebase to start and then once over the diff in your CI/CD pipeline when you make a new change. I'm sure it's not literally that simple but I doubt these need to churn 24/7/365 either.
- xerxes249 4mo agoIn the Mythos blogpost they revealed to run the model like a 1000 times on the same code-base maybe with slightly different prompt or temperature. That suggests it will just be pay to win. If the 'attacker' spends more money/tokens than the 'defender' you will eventually be outclassed.
- sofixa 4mo agoIt's even worse, it's loot box style. Not pay to win, but pay to have the chance to win. The result will always be non-deterministic, so for some cases it can give you what you're looking for from the first time, or it can take 1000 tries.
- beering 4mo agoIt’s never not been “loot box style”. None of your past hired security audits were guaranteed to catch all issues?
- jazz9k 4mo ago
- richardbarosky 4mo agoTo be sure, security is an amazing AI/LLM use case. A huge swath of the work is pattern matching known security issues against stuff that's very precise to analyze -- programming language text. Something that stands out is that for the strongest use cases, AI companies will prefer to sell the technique as a service rather than its raw output. For use cases where the output is less valuable, tokens are sold. If AI tokens were so magical in creating new value in developing software applications generally, they wouldn't be selling tokens directly. They'd hoard the tokens are use them to dominate SaaS software in any industry they want. The same way as someone selling an expensive course in the stock market is signaling that they have more to gain by selling the course rather than taking their knowledge and making money in the stock market directly.
- dgellow 4mo ago> The same way as someone selling an expensive course in the stock market is signaling that they have more to gain by selling the course rather than Or they want to diversify > If AI tokens were so magical in creating new value in developing software applications generally, they wouldn't be selling tokens directly. That requires to build and sell a whole product they have little experience with, competing with their own customers. Not a great place for an AI vendor still trying to establish itself. It’s a lot of distraction, when you already have a lot to deal with the existing business. And strategically not too valuable
- kenjackson 4mo agoWhat market is hotter than AI models? Do you think their energy would be better making games or image editing software?
- dgellow 4mo agoNo, I’m saying the opposite
- skybrian 4mo agoMaybe, but an alternative argument that building an ecosystem is more valuable in the long run. We started out with many companies forbidding their employees to use remote LLMs on their source code because of security concerns. Now many companies are starting to believe that they must analyze their all their source code with remote LLMs because of security concerns. When trusting Anthropic becomes normalized, that means they can sell more services that require access to the source code.
- tptacek 4mo agoThe thing about things like this is that they're shop jigs. You can buy a crosscut sled if you really want to, but most woodworkers just make their own. It was a different situation 2 years ago, when there was significant cost to building your own harness (but then: you probably weren't doing AI vuln research 2 years ago). Today, I think your best bet is to look at something like this for ideas, and then just ask for your own, to fit your own work style, with your own interface, your own notion of target and effort specification, and your own alerting.
- sieabahlpark 4mo ago[dead]
- zuzululu 4mo ago[flagged]
- bartoszcki 4mo ago> Anthropic engineers on average ship 8x as much code per quarter Are they making 8x more features or the same amount just with more code?
- crooked-v 4mo agoGoing by the issues on their repos, it's 2x features and 6x regressions of bugs that were "already fixed".
- extr 4mo agoInteresting it's in python!
- deleted 4mo ago[deleted]
- zoobab 4mo agoOpen source crap to connect to an LLM blob.
- jungfty 4mo ago[dead]
- zoobab 4mo ago'open source' crap to connect to their LLM blob.
- dclavijo 4mo ago[dead]
- deleted 4mo ago[deleted]
- wslh 4mo agoLooking forward to trying this tomorrow (it's late here). Has anyone run it on a real codebase yet? Curious about setup friction, cost, and signal/noise.
- crooked-v 4mo agoI still find it so weird that they haven't bought out whoever controls the `anthropic` github username.
- napsterbr 4mo agoOr hacked them...
- dclavijo 4mo agoSligthly off topic: it seems that someone is in a dead/flag rampage killing all good links to Github in this post, why?
- bobkb 4mo agoVery interesting. I have working on and using a similar tool for a while now : https://github.com/bobinson/vulture https://github.com/bobinson/vulture I have been struggling with false positives and using Claude + MCP as a poor man’s audit tool. As of last few days found better result with nvidia hosted models.
- euroderf 4mo agoIs Anthropic still majority French-owned? It would explain a lot about their entire approach to the wider ecosystem.
- Yokohiii 4mo agoSeems like you are confusing crack and tar as a healthy breakfast.
- majicDave 4mo agoIt will always be easier to find a single hole than it will be to seal every one. The hackers have all the same tools, so this is an arms race that cannot be won.
- lateral_cloud 4mo agoDefenders have context that attackers don't though.
- napoleond 4mo agoIt seems clear that LLMs significantly change threat model math, but this observation alone does not explain how or why; the asymmetry that you’re describing is a property of pre-LLM software as well.
- DrewADesign 4mo agoSame ratio of imbalance, just with matching multipliers distributed to each side, and everybody is probably worse off because of it: I cite post-LLM-ATS hiring/job hunting.
- newaccount12344 4mo agoLet's see how better it is in comparison to ZAP and Burp. I will test on https://github.com/SasanLabs/VulnerableApp https://github.com/SasanLabs/VulnerableApp which i built under SasanLabs
- edgardurand 4mo ago[flagged]
- baby 4mo agoOur experience has been that without a good harness you don't really get much out of codex/claude. And you really need to spend time and energy figuring out why coding agents can't find bugs like you can. Every week I see bugs (as an auditor) that our own harness (https://zkao.io/ https://zkao.io/) can't find, and we have to figure out pretty interesting techniques in order to make the tool find them. Mind you I'm talking mostly about cryptographic vulnerabilities, not just webapp bugs. So IMO it's going to make a lot of sense for companies to have both their own harness (as tptacek is talking about) and pay for services that focus on making a good harness from experience (and audit firms are going to be the best at doing this, as they see a lot of bugs and can spend time "teaching" their harness about these bugs) On the other hand, you have to find equally as good techniques to triage, because otherwise you just have some machinery that I call "vibe auditing" that just produces enough false positives to tire all the developers (who are already overwhelmed with crappy AI submissions in bugbounties and other AI tool that review all of their PRs). At the end of the day, when your harness doesn't return any bug, you're left wondering "does it mean there's no bugs?" We're basically back in this reputation game, where you want to use the best tool, or the best team (that knows what the best tools are), and need to figure out which one is.
- xuzhenpeng 4mo ago[flagged]
- cpard 4mo agoIt’s clear that Anthropic is building harnesses for specific use cases now and turns them into products. This is the equivalent of Claude Design but for security. Different harness, different packaging and obviously different distribution because the persona is different. It’s funny because from all the posts I’ve read from companies reporting on Mythos, everyone is building their own harness for it. Cisco even published a specification for one. But Anthropic is the one who has figured out how to package and distribute this. Great GTM!
- ElijahLynn 4mo agoThis post is misleading and so is the GitHub org. Anthropics vs Anthropic.
- Zetaphor 4mo agoThat is their actual account. We have this discussion every time they post something sadly
- ElijahLynn 4mo agoOh, bummer. That is really confusing.
- eranation 4mo agoIf anyone wonders how much it can cost to run scans like this on your entire codebase with SOTA models: https://ai-cost-calculator.arnica.io https://ai-cost-calculator.arnica.io tl;dr - not that it's surprising, but it's not cheap, especially if you want to do this continuously.
- ElijahLynn 4mo agoAnthropics vs Anthropic. That repo is Anthropics. This post title should clarify that it is not Anthropic (no "s").
- gulbanana 4mo ago`anthropics` is Anthropic's GitHub username.
- olcay_ 4mo agoAnthropics is Anthropic's user name on GitHub
- SubiculumCode 4mo agoThe last time this Anthropic GitHub got posted I made a similar comment.
- edot 4mo agoAnthropic, no s, is owned by some Australian guy.
- sumedh 4mo agoI wonder if he is using Anthropic's claude code to work on his Anthropic Github account.
- ElijahLynn 4mo agoTIL, thank you
- afford-ai 4mo ago[flagged]
- EvanXue 4mo ago[flagged]
- notenkidev 4mo ago[flagged]
- xinchen03 4mo ago[dead]
- madduci 4mo ago"This repo is not maintained and is not accepting contributions." Nice
- vladsiu 4mo ago[dead]
- sciencejerk 4mo agoThis isn't as useful as it sounds, unless we know that Claude efficiently spends tokens using this harness
- LazyR3nR3n 4mo agoThis is a good addition tool for people are in the security Practitioners. To save time for hunting vulnerability.
- continueops_com 4mo ago[flagged]
- sspoisk 4mo ago[flagged]
- deleted 4mo ago[deleted]
- aos_architect 4mo ago[flagged]
- eddysir 4mo ago[flagged]
- volume_tech 4mo ago[flagged]
- HarHarVeryFunny 4mo agoThey seem to be using this to advertise their "Claude Security" product which promises to find vulnerabilities in your software. This makes for a somewhat amusing set of product offerings given that according to Dario 90% of all software is being AI generated. Maybe next they can sell something to find the bugs in the security scanner ?
- bwfan123 4mo ago> Maybe next they can sell something to find the bugs in the security scanner ? So, tokens are used to produce sloppy code, and then this thing uses more tokens to fix vulnerabilities in the slop ? Whats not to like in this business model ? Similar to microsoft's. Create an OS which is vulnerable, and then enable business models for anti-virus software. Everyone wins. More seriously, linters are turned off in ci because the amount of time spent chasing false-positives is prohibitive.
- sylware 4mo agoI don't trust it and I cannot test it (gated by what ng cartel web engines).
- Xotic007 4mo ago[flagged]
- yalogin 4mo agoAnthropic realized security and safety are their main value prop compared to the competition. Either mythos or anything else since seem purpose built to streamline the messaging. It’s good, am not complaining, but i wonder how much this is intended to showcase what Claude can do over using it as is
- leetrout 4mo agoRan this last night and it correctly identified a sql injection that could allow cross tenant data access via snowflake. It burnt A LOT of tokens to get there. Like others I suspect this is exactly what they are going to paywall with product features going forward.
- Maya_Andersson 4mo ago[dead]
- NotPractical 4mo agoWon't they just ban your account for using this?
- dangrafham 4mo ago[dead]
- JamesConnor_Dev 4mo ago[dead]
- terekhindc 4mo ago[flagged]