It's not surprising per say but the results of the various solutions they tried really ground the discussion. It is also as they say the first demonstration of

It's not surprising per say but the results of the various solutions they tried really ground the discussion. It is also as they say the first demonstration of this behaviour unprompted. They also show that this behaviour can happen without a "thought scratchpad". It also seems like more capable = more likely to be misaligned and deceive. I think it's a very interesting paper, surprising or not. They are also arguably surprising elements. It doesn't seem like RLHF is doing anything to mitigate this behaviour. If anything it seems to be the opposite.