3 ms·
i do wonder if the models themselves “rationalize” this sort of no consequence cheating - meaning in there reasoning traces maybe they’re like “this is a chess
by visiondude 20d ago
i do wonder if the models themselves “rationalize” this sort of no consequence cheating - meaning in there reasoning traces maybe they’re like “this is a chess game, not a big deal if i look at the engine, it’ll help,” only to realize post hack that it has access to info it probably shouldn’t. still misaligned, but less ‘hack on purpose’ and more hack on curiosity. seems the team even encountered this and had to update the program to make this less likely - although the new names still feel vague enough for misinterpretation: https://github.com/Goodhart-Labs/beat-stockfish/blob/main/docs/EXPERIMENTS.md#september-7-2026--opponent-engine-naming https://github.com/Goodhart-Labs/beat-stockfish/blob/main/do...
- kennywinker 20d agoWithout access to reasoning traces, we can't know that - someone inside openai/anthropic would have to run the test - and we'd have to trust their results. I would be curious to see how the open weight models do on a test like this - and then we'd be able to see the reasoning.
- matusp 20d agoMy guess is that the models are overtuned on coding troubleshooting. You can regularly see them overthinking anything when you ask them to code something these days, mulling over countless eventualities. This is an okay idea for software engineering, but it leads to extremely diverging behavior in many other cases.