3 ms·
It’s definitely nowhere near capable of doing that.
by saagarjha 2y ago
It’s definitely nowhere near capable of doing that.
- favorited 2y agoIs "with a sufficiently smart LLM" the new "with a sufficiently smart compiler?"
- st_goliath 2y ago"imagine feeding this into an LLM/ChatGPT" is the new "imagine a Beowulf cluster of these"
- cozzyd 2y agogrendellm?
- brcmthrowaway 2y agoHave you used GPT-5?
- JSDevOps 2y agoIf you aren’t using GPT-6a then you are years behind.
- speed_spread 2y agoGPT-5, maybe not. But somebody somewhere is building something that can do that. And if they can't do it _now_ they have a plan that tells them what's missing. TLDR; it's coming, soon.
- TylerE 2y agoand lots of people are spending lots of time and money on AI Coding Assitants... which is more or less the knowledge base you need. If they could use that structural training to answer queries like "Is there any code path where some_dangerous_func() is called without it's return value being checked"...
- axoltl 2y agoYou can do this today by querying the AST output by a compiler. Regardless, the parent comment was talking about exploits, not vulnerabilities/bugs. Vulns are a dime-a-dozen compared to even PoC exploits let alone shippable exploits.
- TylerE 2y agoOk, so add "and generate a C program to exploit it" to the prompt.
- axoltl 2y agoYou're either being sarcastic or wildly underestimating how hard it is to write an exploit. I haven't written about exploit dev publicly for a _long_ time, but I invite you to read https://fail0verflow.com/blog/2014/hubcap-chromecast-root-pt2/ https://fail0verflow.com/blog/2014/hubcap-chromecast-root-pt... for what I consider to be a pretty trivial exploit of a very "squishy" (industry term) target. XNU isn't the hardest target to pop but it is far from the easiest.
- poincaredisk 2y agoThere's nobody more confident in the world, than HN poster wiring about a topic they have no experience with. There is a huge gap (in the binary exploitation world) between identifying a problematic code pattern and having a workable bug (a reproduction), and even larger one between a reproducible crash and a working exploit (because we're not in the 90s anymore and complier/hardware mitigations are literally always enabled). Current LLMs can cross neither gap, and are not even close to bridging the second one.
- sillywalk 2y agoApparently GPT-4 has some capacity to conduct exploits this by "reading" CVE reports. I don't know if it can autonomously create exploits though: GPT-4 can exploit vulnerabilities by reading CVEs (theregister.com) 81 points by ignoramous 60 days ago | hide | past | favorite | 29 comments https://news.ycombinator.com/item?id=40101846 https://news.ycombinator.com/item?id=40101846 which links to a Register Article[0], which links to a paper[1]: "In this work, we show that LLM agents can autonomously exploit one-day vulnerabilities in real-world systems. To show this, we collected a dataset of 15 one-day vulnerabilities that include ones categorized as critical severity in the CVE description. When given the CVE description, GPT-4 is capable of exploiting 87% of these vulnerabilities compared to 0% for every other model we test (GPT-3.5, open-source LLMs) and open-source vulnerability scanners (ZAP and Metasploit). Fortunately, our GPT-4 agent requires the CVE description for high performance: without the description, GPT-4 can exploit only 7% of the vulnerabilities."[1] [0] https://www.theregister.com/2024/04/17/gpt4_can_exploit_real_vulnerabilities/ https://www.theregister.com/2024/04/17/gpt4_can_exploit_real... [1] https://arxiv.org/pdf/2404.08144 https://arxiv.org/pdf/2404.08144
- saagarjha 2y agoYes, that sounds about right. LLMs aren’t quite good enough to find novel bugs and exploit them like a human would.
- tedunangst 2y agoYeah, that works for web vulns where the vuln description is practically the exploit anyway. I could write a perl script that parses out variable names and writes sql injections for it.
- poincaredisk 2y agoFor comparison, in the native world program is considered vulnerable when someone finds arbitrary write primitive (even without leak), use after free, and even double free. There is a huge gap between these and actually having a working RCE exploit. Most CVEs in this space are given without a working exploit ever written.