This tracks with my experience too. For the past few years, I've been using "give me the pinyin for these Chinese characters" as a rough guide for how well a g

This tracks with my experience too. For the past few years, I've been using "give me the pinyin for these Chinese characters" as a rough guide for how well a given model would perform. When GPT4 was released, it absolutely nailed this task; every word was 100% correct, every single time. For reference, even GPT3.5 wasn't great at this. I repeated this last week and it's still getting the phonetic content right, it's now often getting the tones wrong (something which I literally never saw on release). Perhaps changing the prompt would deliver the correct results, but that's not really the point. Something has pretty clearly changed such that old prompts no longer delivering the same results.