5 ms·
I've worked on zero days for decades and friends of mine with great academic pedigrees and deep experience in both AI and security are uniformly frustrated with
by tdullien 2y ago
I've worked on zero days for decades and friends of mine with great academic pedigrees and deep experience in both AI and security are uniformly frustrated with the inability of even cutting-edge LLMs to reason in the way required for security bug detection.
- rfoo 2y ago> to reason Should we expect current LLMs to be able to reason at all? As the paper said GPT-4 maybe make a better scanner. The problem is too many people think all we need is better scanners, which is not new.
- mattgreenrocks 2y agoYup. If you believe vulnerability research is a problem solved by throwing more bodies at it, I can see why LLMs are appealing. I remain skeptical about their efficacy. You need a lot of nonlocal contextual information and very good reasoning skills to be an effective reverse engineer.
- TeMPOraL 2y agoGive it time. GPT-4 is barely 14 months old. Things might look different in a year or two.