Shameless self-plug here, however two years ago I attempted a self study project following a similar intuition: https://maxdunhill.medium.com/how-effective-are

Shameless self-plug here, however two years ago I attempted a self study project following a similar intuition: https://maxdunhill.medium.com/how-effective-are-transformers-at-detecting-vulnerable-code-f23ed443c017 https://maxdunhill.medium.com/how-effective-are-transformers... And created a Hugging Face repo of known vulnerabilities if anyone else wants to work on a similar project (link in blog post). My project was a lay person’s not especially successful attempt to fine-tune a BERT-based classifier to detect vulnerable code. Having said this, a main takeaway echoes simonw’s comment: “LLMs are great at pattern matching, so it turns out feeding in a pattern describing a prior vulnerability is a great way to identify potential new ones.” Given majority of vulnerabilities stem from memory misallocation, it seems that an LLM would most consistently find misallocated memory. Useful, though not the most complex vulnerabilities to weaponise. It seems the next frontier would be for an LLM to not only identify previously unidentified vulnerabilities, but also describe how to successfully daisy chain them into an effective vulnerability exploitation. Said differently, giving an LLM a goal like jail breaking the iOS sandbox and seeing how it might approach solving the task.