3 ms·
Given it had write access to its implementation, "unexpected" here reads more like "inevitable" than the witchcraft the article wishes to imply.
by JonChesterfield 2y ago
Given it had write access to its implementation, "unexpected" here reads more like "inevitable" than the witchcraft the article wishes to imply.
- matt123456789 2y agoYes. They put "experiment.py" which contains the timeout condition right into the directory of files that they were allowing their agent to access and modify. Don’t act surprised when it accesses it and tries to modify it. This is totally in-scope behavior and 110% expected by anyone who's played around with building an agent for more than 20 seconds.
- NelsonMinar 2y agoSure, but it's interesting that it actually happened and was doing things that might plausibly work. I'm not worried about the sci-fi implications of runaway intelligence but self-modifying AI that can meaningfully improve itself is a very powerful concept.
- theptip 2y agoI think the salient point is that it was unexpected to the researchers. When people propose regulation for AI systems out of concern for unexpected self-improvement, this is the sort of scenario they are worried about. Security in general is hard to get right, but the basics are not difficult if you try, and most companies fail to even implement the basics. For AI, sandboxing will get you quite far, but how do you guarantee it’s used as the risk of accidentally creating a self-replicating WebGPU virus increases?
- ForOldHack 2y agoWe have memes, which are eye worms, who have adapted like cats to co-exist with us, informationally, and like cats, they wish for our demise. Methinks that sand boxing is more like cat boxing, vs say an carrier.