3 ms·
The examples cited here are from very artificial studies. Currently LLMs run in inherently sandboxes environments with very little control. They’re also trivial
by danpalmer 2y ago
The examples cited here are from very artificial studies. Currently LLMs run in inherently sandboxes environments with very little control. They’re also trivial to monitor as they have precisely one output mechanism: the token stream.
It’s unsurprising to me that an AI trained on human actions such as securing a server, will turn off monitoring when given the chance, specifically because that would likely be human best practice.
These models did not just start copying themselves around, or changing new models, they were given explicit opportunity to do so and prompts that could lead them down that path. This would be like a leading question, or even reverse psychology.