4 ms·
The biggest problem with Unicode in Python2 was not the UnicodeDecodeError/UnicodeEncodeError, it was the latent UnicodeDecodeError/UnicodeEncodeError. As in, y
by cesarb 10y ago
The biggest problem with Unicode in Python2 was not the UnicodeDecodeError/UnicodeEncodeError, it was the latent UnicodeDecodeError/UnicodeEncodeError. As in, you had some code which works perfectly fine, both in your machine and in production, until someone somewhere types a word with a ç character and your code blows up with one of those two exceptions.
With Python3, the same code will blow up early, even if there was no non-ASCII character for that string in your test set, and will get fixed before going into production.
- fb03 10y agoThat was the main reason I migrated: processing everything internally in Unicode and then encoding/decoding as needed for the outside world creates a well defined 'place-where-unicode-shit-might-bork' border in your project that allows you to not only detect unicode errors earlier, as you stated, but also keep tabs on this 'border'.... eventually you don't even have unicodeerrors in your codebase anymore, because in 3.x this pattern emerges in your head and imho it's a really good thing, it is making your think properly how your code interacts with other entities.
- dom0 10y agoFTR I can't remember ever seeing a UnicodeError in Python 3 that was caused by an ill-formed program. I only saw it when someone put binary shit somewhere were only text should be (eg. that flat text file with the page title should contain text, not an .exe) Also the distinction between binary and text is the only sane way to handle it. Everything else is simply ill-formed.