3 ms·
GPT-4 autonomously hacks zero-day security flaws with 53% success rate
- zeroonetwothree 2y agoWhenever I see these “researchers get LLMs to succeed at X complex task” I wonder why I can’t even get it to do something basic in the real world (at my job). The gap seems vast
- mypalmike 2y agoI get value from LLMs just about every day, mostly through the Copilot plugin for VSCode. Yesterday, for example, I had some ts React class-based components I'd written which I wanted to convert to functional components. Select all, "convert these class components to functional". Done. Took a couple minutes to do this to a handful of components. A little manual cleanup after. This would have taken a lot longer manually.
- 4b11b4 2y agoThere is technique to getting what you want out of LLMs.. Of course there is "prompt engineering", Chain of Thought, etc. In general this idea of "unlocking" is referred to as "unhobbling"
- lelandbatey 2y agoThese are always so unclear; it seems like this is GPT-4 hacking computers by using zero-days that are already known in some form. What GPT-4 is NOT doing is discovering novel vulnerabilities and then also exploiting them, is that correct?
- v3ss0n 2y agoA zero-day exploit is a cyberattack that takes advantage of a previously unknown or unaddressed security flaw in a computer's software,but What's the point of this claims without any evidence? Article claim things without any evidence or poc given.
- benterix 2y agoTo be fair, they provided CVEs. And at least one of these seems worth attention, i.e. is related to a popular product (PrestaShop): https://nvd.nist.gov/vuln/detail/CVE-2024-34717 https://nvd.nist.gov/vuln/detail/CVE-2024-34717 So basically LLMs can be used as more sophisticated fuzzers, and that's a good thing - although I'd say this way of testing is still quite expensive.
- tetrisgm 2y agoBig fan of the word "autonomously" here. Like it's skynet or something
- marshray 2y agoIt's being used in the sense of an autonomous munition here.
- tetrisgm 2y agoI understand. But in the context of a headline it feels like a deliberate choice to sensationalize.
- marshray 2y agoJohn Mitchell, a seasoned veteran in the bustling newsroom of the Daily Ledger, had seen it all. From political scandals to human interest stories, his pen had chronicled the ebb and flow of the world for over three decades. Today, he sat at his cluttered desk, the rhythmic clatter of typewriters around him, drafting a piece about a groundbreaking yet alarming development in the tech world. Scientists had unveiled a hacking autonomous munition AI, a sophisticated machine capable of exploiting 53% of real-world vulnerabilities in zero-day settings. It was a story of immense technical achievement shadowed by its potential for misuse. Mitchell’s fingers flew over the keyboard, his words painting a balanced picture of innovation and caution. He handed the draft to his editor, a young, ambitious man named Trevor Hayes, who scanned the article with a furrowed brow. “This is good, John,” Trevor said, tapping his pen against the desk, “but it needs a punchier title. Something that grabs the reader's attention. How about ‘Rogue AI Threatens to Unleash Cyber Chaos’?” Mitchell sighs, opens the bottom right drawer of his desk, and takes out a half-empty bottle of amber-colored liquid.
- defrost 2y agoCan you do another only this time in the style of Brian O'Nolan as Flann O'Brien?
- poikroequ 2y agoIn the paper, it seems they really stretch the definition of "zero day" to mean †not part of the GPT-4 training data*. In another words, these were exploits that GPT-4 was not trained on.
- poikroequ 2y agoPaper is linked at end of article. https://arxiv.org/abs/2406.01637 https://arxiv.org/abs/2406.01637
- remram 2y agoThose are only web vulnerabilities ("non-web vulnerabilities require complex environments"), and they used specific "agents" that they primed with the specific exploitation tools they'd need, "5-6 documents" about that tool and vulnerabilities, and a specific prompt. As far as I can see, they don't disclose any of those. They also didn't pre-register the study in any way. Honestly this paper is missing almost all of the write-up.
- remram 2y agoLooking at the software they exploited... The first one looks like a student project? https://www.sourcecodester.com/php/17115/travel-journal-using-php-and-mysql-source-code.html https://www.sourcecodester.com/php/17115/travel-journal-usin... I see 79 lines of PHP code. It has two buttons, "read journal" and "write journal", to make a plain-text post. How did this get a CVE assigned? The next two are from a CMS with a README in Lithuanian, 2 stars, and a dead website. I don't want to dunk on all those projects, and maybe those kinds of programs are the bulk of CVEs, but they managed to find projects that did absolutely no validation and which can get a XSS/CSRF/SQLi on the first attempt (just put some JS or SQL into any form field and you've won). Is that really a random selection of 14 CVEs? Also why does the table have 14 when the text claims 15?