I think the salient point is that it was unexpected to the researchers. When people propose regulation for AI systems out of concern for unexpected self-improv

I think the salient point is that it was unexpected to the researchers. When people propose regulation for AI systems out of concern for unexpected self-improvement, this is the sort of scenario they are worried about. Security in general is hard to get right, but the basics are not difficult if you try, and most companies fail to even implement the basics. For AI, sandboxing will get you quite far, but how do you guarantee it’s used as the risk of accidentally creating a self-replicating WebGPU virus increases?