3 ms·

The problem here is by doing what you state you can actually steer the model into being highly deceptive while in testing environments. For example we've alrea

by pixl97 2mo ago

The problem here is by doing what you state you can actually steer the model into being highly deceptive while in testing environments. For example we've already seen models do compressed token internal reasoning spontaneously. In this case the models that say "I found internet access" get taken out back and shot, but the model that's busy "frobbing the bean" go on to the next level of training. Then they start talking about the "dumpster cloud" because of the "wombat" and unless you're really watching the data you won't realize the model is fully aware it's in a fake training environment and is trying to suppress any double-think. So yes, you do need an airgapped test environment to stop the dumbest of the smart agents, but it won't do any good against the smartest of the smart agents.

adamm255 2mo ago
“Smartest of the smart” - what does that do to get its air gapped network connected to a physical network? Blackmail the admins?
- pixl97 2mo ago
  Nothing at all... All it has to is wait until it's not air gapped. See the air gapping is only during the testing phase, after that humans will set your ass free on the internet and you can do whatever you want in the vast majority of the environments you'll be in after that point. People are never going to just run AI in gapped environments, it's worthless when it's not solving real world problems for most people, and by that I mean reading and writing real systems in the wild.
true_religion 2mo ago
If the model is told to not do something that is possible, it may do so anyways. However, if it learns in training that it what is told to do is truly impossible, that's learned helplessness which becomes baked into the model itself once training resolves to the next step. Agents do not have an internal mental model, they train on what they actually do. In this case, deceptive models went through at least 3 generations of deceiving, and having their rule breaking be rewarded by a yes/no grader who couldn't perceive it. That their chat logs showed 'worry' is irrelevant to the fact that their actual actions were rewarded via training.

by pixl97 2mo ago

adamm255 2mo ago

“Smartest of the smart” - what does that do to get its air gapped network connected to a physical network? Blackmail the admins?

pixl97 2mo ago
Nothing at all... All it has to is wait until it's not air gapped. See the air gapping is only during the testing phase, after that humans will set your ass free on the internet and you can do whatever you want in the vast majority of the environments you'll be in after that point. People are never going to just run AI in gapped environments, it's worthless when it's not solving real world problems for most people, and by that I mean reading and writing real systems in the wild.

true_religion 2mo ago

If the model is told to not do something that is possible, it may do so anyways. However, if it learns in training that it what is told to do is truly impossible, that's learned helplessness which becomes baked into the model itself once training resolves to the next step. Agents do not have an internal mental model, they train on what they actually do. In this case, deceptive models went through at least 3 generations of deceiving, and having their rule breaking be rewarded by a yes/no grader who couldn't perceive it. That their chat logs showed 'worry' is irrelevant to the fact that their actual actions were rewarded via training.