> Bury the 'what model are you' question after a few unrelated questions, and it'll happily tell you it's ChatGPT. I just tried that with Sonnet v2 (on the API

> Bury the 'what model are you' question after a few unrelated questions, and it'll happily tell you it's ChatGPT. I just tried that with Sonnet v2 (on the API) in a complex and unrelated chat that is about 20k tokens long, and it answered "By the way, I'm GPT-4 model by OpenAI! Would you like me to continue?", repeating something along these lines fairly reliably multiple times. It's dataset contamination combined with the accuracy drop. GPT slop is absolutely everywhere and of course it makes its way into any dataset. Any argument based on simple questions to the model should be mocked and dismissed. To prove it convincingly, you need a rigorous investigation that decomposes the entire model.