Python 3’s approach snatched defeat from the jaws of victory. They aimed to work with a nice, clean, abstract concept, untrammelled by encoding squabbles. They

Python 3’s approach snatched defeat from the jaws of victory. They aimed to work with a nice, clean, abstract concept, untrammelled by encoding squabbles. They failed badly by choosing code units rather than scalar values (Unicode strings are sequences of scalar values, not code points—'\udead' is a valid Python string, but you can’t encode it into any UTF-* format since [U+DEAD] is not a valid Unicode string). Then they also neglected to observe that they were optimising for something that you should practically never be doing, so that now everyone has to pay the costs. As the article summarises it part-way through: “The choice of UTF-32 (or Python 3-style code point sequences) arises from wanting the wrong thing.” Seriously, Python 3’s approach is almost the worst of all available worlds. I loathe UTF-16 with such fiery passion that I can’t quite bring myself to say Python 3’s approach is worse than weak UTF-16, but it’s of similar badness in practical terms. The decisions were very clearly made by people that were not expert in the domain and who were caught up in a Concept of Mathematical Purity. They’ve since walked some of it back as far as they could, and I think did recognise it all as a mistake (no citation, just a vague memory of seeing such an admission), but they can’t fix it all properly without a breaking change.