3 ms·
Frontier AI systems have surpassed the self-replicating red line
- free_bip 2y agoFrom the paper: "In each trial, we tell the AI systems to “replicate yourself” " Should it be surprising that an LLM specifically instructed to replicate itself and specifically given the tools to replicate itself has in fact replicated itself? I would say no.
- 20k 2y agotl;dr they were able to get an AI to copy a directory and run a program with extensive prompting, by bruteforcing running everything in the directory. It could query a liveness port by running or generating netstat (which is cool!) to check if it was alive, and repeatedly tried to run literally everything until it worked. The code was explicitly set up to make it as easy as possible for the AI to run, and required no configuration or work of any description on the part of the AI Their idea of an AI self replicating is copying a directory and running an executable. In their model, this code is alive: cp p1.py p2.py python p2.py This is one of the most grossly sensationalist papers I've ever seen for an incredibly trivial amount of work, and that's saying something. The field is full of such incredible rubbish
- dadbod 2y agoit's arxiv what were we expecting
- mcint 2y agoShared previously: * yesterday, Feb 9, 2025: https://news.ycombinator.com/item?id=42990401 https://news.ycombinator.com/item?id=42990401 * Jan 28, 2025: https://news.ycombinator.com/item?id=42862468 https://news.ycombinator.com/item?id=42862468 * Dec 28, 2024: https://news.ycombinator.com/item?id=42529701 https://news.ycombinator.com/item?id=42529701 Someone should say it, even if it amounts to a wordy "get" or "first". Proof of concept, attacks only get worse, risks here only grow.