It's not surprising news. It's obvious news. When I was making those claims, I hadn't tested it: I was making purely theoretical arguments, based on having glan

It's not surprising news. It's obvious news. When I was making those claims, I hadn't tested it: I was making purely theoretical arguments, based on having glanced at the relevant papers in the GPT-2 days. That GPT models are bad at translation, and will always be bad at translation (while, perhaps, "improving" where they're overfit on specific benchmarks), is obvious to anyone with even a cursory understanding of how they work.