3 ms·
I think the key insight from this is: > We also feel that this variant-analysis task is a better fit for current LLMs than the more general open-ended vulnerab
by simonw 2y ago
I think the key insight from this is:
> We also feel that this variant-analysis task is a better fit for current LLMs than the more general open-ended vulnerability research problem. By providing a starting point – such as the details of a previously fixed vulnerability – we remove a lot of ambiguity from vulnerability research, and start from a concrete, well-founded theory: "This was a previous bug; there is probably another similar one somewhere".
LLMs are great at pattern matching, so it turns out feeding in a pattern describing a prior vulnerability is a great way to identify potential new ones.
- ngneer 2y agoThanks for highlighting a key aspect. It may be informative to see all the ways in which the LLM got it wrong along the way, before arriving at the cherrypicked example. For instance, was this the only pattern flagged as a potential bug in the process? That said, I am surprised at how far pattern matching can get you. Certain exploitation requires arithmetic. If LLMs struggle with simple arithmetic, then it is unclear why we would expect them to do well with these. In the GPZ example, it does not appear that the assistant can do the arithmetic or realize the negative offset and its implication. Rather, it looks like pattern matching that mimics static dataflow analysis in order to arrive at the assertion failure. "To cause the assertion failure, we need a constraint on a column with index greater than 3 or smaller than 1."
- simonw 2y agoI would be shocked if this process didn't kick out hundreds of false positives. In this context I think that's fine - zero-day vulnerabilities are rare, so security researchers should expect to browse through all sorts of mis-leads on the way to finding one.