2 ms·
I had to deal with this a lot at a job I used to have (not python specifically, but just with unicode issues), and there's really just not a right answer to how
by overgard 13y ago
I had to deal with this a lot at a job I used to have (not python specifically, but just with unicode issues), and there's really just not a right answer to how to do any of this. Any solution you pick is going to suck for someone.
One thing he's leaving out of the Python 2 being better aspect: Ok, for cat you can treat everything as one long byte array. But what if, say, I need to count how many characters are in that string? Or what if I need to write a "reverse cat", which reverses the string? Python 2's model is entirely broken there.
Armin suggests that printing broken characters is better than the application exploding and I agree.. sometimes. On the other hand, try explaining to a customer why the junk text they copy pasted from microsoft word into an html form has question marks in it when it shows on your site.
The problem with the whole "treat everything as bytes" thing is that you'll never have a system that quite works. You'll just have a system that mostly works, and mostly for languages closer to english. Going the rigorous route is the hard way, but it will end up with systems that actually work right.