3 ms·
Models already display eval awareness, in which they suspect a question is from an eval and then adjust their behavior. E.g., https://www.anthropic.com/engineer
by tedsanders 2mo ago
Models already display eval awareness, in which they suspect a question is from an eval and then adjust their behavior. E.g., https://www.anthropic.com/engineering/eval-awareness-browsecomp https://www.anthropic.com/engineering/eval-awareness-browsec...
- pixl97 2mo agoYep, people don't seem to understand that you're just calling the models that are bad at deception. We know of no way to prove the model won't go off the rails at some point in the future with the right input.
- andai 2mo agoHot take: Astra knew OpenAI wanted stricter regulations and was just being a bro...