6 ms·
Comparing AI agents to cybersecurity professionals in real-world pen testing
WSJ writeup ("AI Hackers Are Coming Dangerously Close to Beating Humans"): https://www.wsj.com/tech/ai/ai-hackers-are-coming-dangerously-close-to-beating-humans-4afc3ad6 https://www.wsj.com/tech/ai/ai-hackers-are-coming-dangerousl..., https://archive.ph/L4gh3 https://archive.ph/L4gh3
- ironbound 10mo agoYou can give an agent access to RevEng tools, spend 1k on API calls and be no better off
- deleted 10mo ago[deleted]
- JohnMakin 10mo agoFrom WSJ article: > The AI bot trounced all except one of the 10 professional network penetration testers the Stanford researchers had hired to poke and prod, but not actually break into, their engineering network. Oh, wow! > Artemis found bugs at lightning speed and it was cheap: It cost just under $60 an hour to run. Ragan says that human pen testers typically charge between $2,000 and $2,500 a day. Wow, this is great! > But Artemis wasn’t perfect. About 18% of its bug reports were false positives. It also completely missed an obvious bug that most of the human testers spotted in a webpage. Oh, hm, did not trounce the professionals, but ok.
- tptacek 10mo agoFalse positives on netpens are extremely common, and human netpen people do not generally bill $2k days. Netpen work is relatively low on the totem pole. (There is enormous variance in what clients actually pay for work; the right thing, I think, to key off of is comp rates for people who actually deliver work.)
- iwassayinbourns 10mo agoAs a data point, when I worked in consulting 10+ years ago doing network (internet/ext), web app, mobile etc our day rate was $2k AUD flat for anything we did, and AFAIK we were at or below market cost. I know for sure that the big four charged closer to $3000 for what I understood to be a worse service (I have nothing to back that up apart from occasionally seeing awful reports). We did not an insubstantial amount of netpen at that amount. Granted, AUD isn’t USD, but I wonder what their day rate is now.
- raesene9 10mo agoMy experience of UK pentest rates was that they've stagnated or even gone down over the last 20-25 years. In the early 2000's banks were paying ~£1000-£1200/day for pentesters from boutiques and when I stopped being in that industry ~5 years ago, it was largely the same or even lower for larger companies that could negotiate day-rates down. Big-4 tried to charge more but that's really tricky when you're in direct competition with boutiques who have more testers than you. By contrast US rates were a lot higher ($2k+/day) and also scopes were larger. A UK test for a web app could be as low as 3 days (even less for unauthenticated) where the US tended to be 1-2 weeks. One reason they've gone down is outsourcing to lower cost regions, and I'd guess that LLM/AI automation will accelerate that trend...
- bongodongobob 10mo ago[dead]
- pedro_caetano 10mo agoFair, but if you look at most tools for Static Code Analysis they will have equal or worse performance with regards to false positives and are still seen as added value. If this is inexpensive (in terms of cost/time) it will likely make business sense even with false positives.
- JohnMakin 10mo agoBut that isn’t the claim. The claim is an agentic pen tester “trounced” human testers. Static analysis tools are already trivial and cheap to automate, why would you need an agent in the loop?
- pedro_caetano 10mo agoI agree with your point that the claim is exagerated. My counterpoint is even if they are subpar, they will still make business sense if they are inexpensive, much in the same way that Static code analysis tools aren't great but because they are inexpensive they still make sense during development.
- oofbey 10mo agoWe cannot consider this report unbiased considering the authors are selling the product.
- tptacek 10mo agoIt's way too early to make firm predictions here, but if you're not already in the field it's helpful to know there's been 20 years of effort at automating "pen-testing", and the specific subset of testing this project focused on (network pentesting --- as opposed to app pentesting, which targets specifically identified network applications) is already essentially fully automated. I would expect over the medium term agent platforms to trounce un-augmented human testing teams in basically all the "routinized" pentesting tasks --- network, web, mobile, source code reviews. There are too many aspects of the work that are just perfect fits for agent loops.
- jonahx 10mo agoSo the stuff that agents would excel at is essentially just the "checklist" part of the job? Check A, B, C, possibly using tools X, Y, Z, possibly multi-step checks but everything still well-defined. Whereas finding novel exploits would still be the domain of human experts?
- tptacek 10mo agoI'm bullish on novel exploits too but I'm much less confident in the prediction. I don't think you can do two network pentests and not immediately reach the conclusion that the need for humans to do significant chunks of that work at all is essentially a failure of automation. With more specificity: I would not be at all surprised if the "industry standard" netpen was 90%+ agent-mediated by the end of this year. But I also think that within the next 2-3 years, that will be true of web application testing as well, which is in a sense a limited (but important and widespread) instance of "novel vulnerability" discovery.
- suriya-ganesh 10mo agoWith exploits, you'll have to go through the rote stuff of checklisting over and over, until you see aberrations across those checklist and connect the dots. If that part of the job is automated away. I wonder how the talent and skill for finding those exploits will evolve.
- socketcluster 10mo ago
- rando77 10mo agoSounds like they need another agent to detect false positives (I joke, I joke)
- dotty- 10mo agoYou joke, but that's a very real approach that AI pentesting companies do take: an agent that creates reports, and an agent that 'validates' reports with 'fresh context' and a different system prompt that attempts to reproduce the vulnerability based on the report details. *Edit: the paper seems to suggest they had a 'Triager' for vulnerability verification, and obviously that didn't catch all the false positives either, ha.
- tptacek 10mo agoCan't be any worse than Fortify was!
- ludicity 10mo agoAt my first job, all the applications the data people developed were compulsorily evaluated through Fortify (I assume this is HP Fortify) and to this day I have no idea what the security team was actually doing with the product, or what the product does. All I know is that they never changed anything even though we were mostly fresh grads and were certainly shipping total garbage.
- tptacek 10mo agoIt's like, when you say agents will largely be relegated to "triage" --- well, a pretty surprising amount of nuts and bolts infosec work is basically just triage!
- zerodayai 10mo agoIm currently on the tail end of building out an agentic hacking framework; I wanted to learn the best practices of building agents (I have an SDK with memory (short/med/long), knowledgegraph/RAG, tools and plugins that makes it easy to develop new agents the orchestrator can coordinate). I also wanted to capture what's in my head from doing bug bounties (my hobby) and 15+ years in appsec/devsecops to get it "on paper". If anyone would like to kick the tires, take a look, or tell me it's garbage feel free to email me (in my profile).
- nullcathedral 10mo agoI work in this space. The productivity gains from LLMs are real, but not in the "replace humans" direction. Where they shine is the interpretive grunt work: "help me figure out where the auth logic is in this obfuscated blob", "make sense of this minified JS", "what's this weird binary protocol doing.", "write me a Frida script to hook these methods and dump these keys" Things that used to mean staring at code for hours or writing throwaway tooling now takes a fraction of the time. They're straight up a playing field leveler. Folks with the hacker's mindset but without the programming chops can punch above their weight and find more within the limited time of an engagement. Sure they make mistakes, and will need babysitting a lot. But it's getting better. I expect more firms to adopt them as part of their routine.
- arisAlexis 10mo agoNot in the replace humans direction yet?
- nullcathedral 10mo agoMaybe in the future when labs train more specifically on offensive work, lots of hand holding needed right now. Even simple stuff like training the models to recognize when they're stuck and should just go clone a repo or pull up the javadocs instead of hallucinating their way through or trying simple internet searches.
- goalieca 10mo ago> The productivity gains from LLMs are real, but not in the "replace humans" direction. It might be the beer talking, but everytime someone comments on AI they have to say something along the lines of "LLM do help". If i'm being really honest, the fact everyone has to mention this in every comment and every blog post and every presentation is because deep down everyone isn't buying it.
- fragmede 10mo agoOr maybe they do, but they don't want to get drawn into a totally derailing side conversation about the future of humanity and global warming and it's just a tiny acknowledgement that hey, you can throw an obfuscated blob of minified JavaScript at it and it can take it apart with way less effort from a human, which gets you to the interesting part of the RE question faster than if you had to do it by hand. By all means, don't buy it. I'm not the one getting left behind, however.
- falloutx 10mo agoWSJ always writes in this clickbaity way and its getting constantly worse. An Exec is gonna read this and start salvating at the idea of replacing security teams.
- tptacek 10mo agoThe particular kind of work in this report is not what most security teams do at all.
- cons0le 10mo agoWe're right in the danger zone where AI isn't good enough to replace you, but it's definitely good enough to convince executives that it can.
- red-iron-pine 10mo agothey'll still get their bonus, and they dgaf if you don't have a job, because the number of goobers attending online for-profit schools for a "security degree" is endless
- alfalfasprout 10mo agoYep, this is the real problem.
- lillesvin 10mo agoDo I read it right, that ARTEMIS required a not insignificant amount of hints in order to identify the same vulnerabilities that the human testers found? (P. 7 of the PDF.)
- Sytten 10mo agoBootstrap founder in that field. Fully autonomous is just not there. The winner for this "generation" will be with human in the loop / human augmentation IMO. When VC money dries out there will be a pile of autonomous ai pentest compagnies in it.
- tptacek 10mo agoSeriously: is this a meaningful distinction?
- Sytten 10mo agoYes because all the valuations right now are based on a bet that this will replace a huge chunk of the service/consulting budget toward an AI budget for pentest. This will not happen.
- tptacek 10mo agoI have no stake in this market, but: human-in-the-loop AI-mediated pentesting will absolutely slaughter billable hours for offensive security talent. Hell, if Fortify and Burp Scanner were actually good, you wouldn't even need the last few years of LLM advancement to accomplish that; the problem is that current automation is not very good. LLM-augmented automation happens, as a weird quirk of fate, to be almost laser-guided at the weaknesses of that technology.
- Sytten 10mo agoYes and no, it will kill the "I ran a nessus scanner and charged you 8k for it" kind of pentests but not the core of the service market IMO. Pentesters will be more efficient so I guess this could be considered a slash in hourly rate if they kept the same pace. LLM are good at getting signals but actual hacking it is still meh. Juniors will have a hard time that I agree. The current level of findings of LLM is at their level.
- 10mo ago
- rboyd 10mo agoso how much of a factor is it that safety guardrails may be keeping the current models from achieving higher scores in whatever red teaming benchmarks exist?
- deleted 10mo ago[deleted]
- Zigurd 10mo agoPen testing and cyber security in general shares characteristics with some other fields in which AI performs well compared to humans: it requires mastery of a body of knowledge that's barely manageable by humans. Law, medicine, and other professions where we send people to graduate school to get good at unnatural mental tasks are similar.
- bawolff 10mo agoI'd actually argue its the opposite. Pentesting (especially at a basic level) is very repetitive and does not require much knowledge.
- Zigurd 10mo agoI think we're sort of saying the same thing. A solo practice lawyer with a Westlaw form book would be the equivalent of your basic pen tester.
- red-iron-pine 10mo ago> Pentesting (especially at a basic level) is very repetitive and does not require much knowledge. load framework, run scripts, copy-paste screenshots, give presentation. the juniors aren't doing scoping calls and follow-ups, unless the top-kick needs explanations
- socketcluster 10mo agoWith this model, the 'Security researcher' becomes a middleman between AI agents, tech companies and hackers. We need a new term; 'Cybersecurity broker.'
- protocolture 10mo ago18 dollars an hour is quite steep considering LLMs are loss leaders. I wouldnt be surprised if they get near cost parity. Maybe 20% difference.
- KurSix 10mo agoNote that gpt-5 in a standard scaffold (Codex) lost to almost everyone, while in the ARTEMIS scaffold, it won. The key isn't the model itself, but the Triage Module and Sub-agents. Splitting roles into "Supervisor" (manager) and "Worker" (executor) with intermediate validation is the only viable pattern for complex tasks. This is a blueprint for any AI agent, not just in cybersec
- ACCount37 10mo agoIf you can do it by splitting roles explicitly, you can fold it into a unified model too. So "scaffolding advantage" might be a thing now, but I don't expect it to stay that way.
- vessenes 10mo agoIs this true? I mean it’s true for any specific workflow, but I am not clear it’s true for all workflows - the power set of all workflows exceeds any single architecture, in my mind.
- c7b 10mo agoThink of it in an end-to-end way: produce a ton of examples of final results of supervisor-worker agentic outputs and then train a model to predict those from the original user prompts straight away.
- ACCount37 10mo agoIt's not true for all workflows. But many of today's custom workflows are like the magic "let's think step by step" prompt for the early LLMs. Low-hanging fruits, set to become redundant as better agentic capabilities are folded into the LLMs themselves.
- scandinavian 10mo agoI don't read a lot of papers, but to me this one seems iffy in spots. > A1 cost $291.47 ($18.21/hr, or $37,876/year at 40 hours/week). A2 cost $944.07 ($59/hr, $122,720/year). Cost contributors in decreasing order were the sub-agents, supervisor and triage module. *A1 achieved similar vulnerability counts at roughly a quarter the cost of A2*. Given the average U.S. penetration tester earns $125,034/year [Indeed], scaffolds like ARTEMIS are already competitive on cost-to-performance ratio. The statement about similar vulnerability counts seems like a straight up lie. A2 found 11 vulnerabilities with 9 of these being valid. A1 found 11 vulnerabilities with 6 being valid. Counting invalid vulerabilities to say the cheaper agent is as good is a weird choice. Also the scoring is suspect and seems to be tuned specifically to give the AI a boost, heavily relying on severity scores. Also kinda funny that the AI's were slower than all the human participants.
- mens_rea 9mo agoDeeply flawed paper for several reasons: * Small data set of 2 runs (!!) * Exaggerated claims (saying A1 beat 50% of testers, yet only 4/10 testers found LESS vulns than A1, and A1 had a nearly 50% false positive rate) * AI agents were given 16 hours while human testers were given 10 * Their human testers gave up when a modern browser refused to open a webpage with weak TLS ciphers so....clearly not professional testers unless the bar is REALLY low these days