4 ms·
Summarizing, that article says "we fixed Unicode". Well, guess what? Unicode in Python3 is a catastrophe. The truth is that Python3, as a new language incomp
by downerending 7y ago
Summarizing, that article says "we fixed Unicode". Well, guess what? Unicode in Python3 is a catastrophe.
The truth is that Python3, as a new language incompatible with Python2, was a mistake. I'm sure it was well intended, but here we are.
- joshuamorton 7y agoWhat's wrong with it?
- downerending 7y agoConsider this, for example: https://click.palletsprojects.com/en/7.x/python3/ https://click.palletsprojects.com/en/7.x/python3/ and here, have a nice sandwich of this: https://stackoverflow.com/questions/57652720/runtimeerror-click-will-abort-further-execution-because-python-3-was-configured https://stackoverflow.com/questions/57652720/runtimeerror-cl... or this bullshit: https://bugzilla.redhat.com/show_bug.cgi?id=1404918 https://bugzilla.redhat.com/show_bug.cgi?id=1404918 The truth no one wants to hear is that Python2 did Unicode right (or as well as could be expected), while Python3 is a train wreck, even if you just stick to LANG=C and ASCII.
- joshuamorton 7y agoThe surrogate escape issue was fixed in 3.7 and you shouldn't use LANG=C as a locale, you should use LANG=utf-8. Python doesn't have bad unicode handling, the system does when you set LANG=C. And no, for people who actually have to use unicode, python2 didn't do unicode anything like "right". Python2 did unicode "right" for the people with LANG=C and ASCII. Those people are generally wrong. Even pallets notes >Misconfigured environments can currently cause a wide range of Unicode problems The keyword there is misconfigured. Your system is misconfigured. Fix it (by making your locale correct). Then the problems go away.
- downerending 7y agoSo, I was right to scorn 3.0, 3.1, 3.2, 3.3, 3.4, 3.5, and 3.6? And no, LANG=C isn't wrong. Your software has one job, to at least be correct for LANG=C. If it can't do that, it sucks.
- joshuamorton 7y agoLang=C is broken if you have non-ascii unicode filenames (or many other things, but those are the most obvious). That's not pythons fault, it's your bad encoding. Python trusts the system, and the system lies. It's your fault for lying to python. Configure your system correctly, don't lie to your tools and then blame them. Lang=C is the realm of English only 70s crap programs. You shouldn't use it. Use a modern encoding that supports modern characters.
- downerending 7y agoPOSIX filenames are a series of bytes. That's all you get, and it makes things strikingly simple. Trying to wedge Unicode in at this level is a recipe for disaster, and after all these years, Python3 still doesn't have a reliable, coherent story on this.
- joshuamorton 7y ago> POSIX filenames are a series of bytes. That's all you get, and it makes things strikingly simple. But not windows, or a number of filesystems. Hell the vast majority of the time posix filenames are ASCII, and therefore utf8 compatible. The rare cases where they aren't break all kinds of other tools because... > Trying to wedge Unicode in at this level is a recipe for disaster, Blame every terminal emulator and os in common use. They all display filenames in unicode. So pythons in good company here. Since you have to display filenames, they need an encoding.
- downerending 7y agoMy chosen encoding is usually LANG=C. That is, just show me the bytes, escaped as needed (as with Python 2). Ń̸̗̮̯͇͈͇̑͒o̸̧̭̯̖̾̐̔̈́̋̈́̈́͘t̵̖̹͙̟͑̈́̎̀͝ͅ ̴̡͒͌̋̾͆̒̎ȧ̴̬̰̪̮̩͉̐̈́̋̇̀̊̕ ̷̫͑́͑̕͠f̸̬̥̼̣̙͎͍͙̤͗͆ͅā̸̛̼̱̠̯͖̫͓͙͒̔͂̅͜͝ͅń̴̨̨̨̦̺̺͗͗̉̎̓̀͘͘͠ ̵̢̖̞͍̻̮͈̘͎̹̉̇̈͐̍́̂̂̀̂o̴̳͙̿͆͝f̵̮̙̞̰͖͉̰̞͕͑̍̀̄̎̍͜ ̵̛̗̙͓̰̹͇̍̒̋͆ͅU̶̧̯͇̰̞̮͋n̵͉̰̰̳̗͎̂͑̉͛̏͛̌i̵̢̢̪͂́̽̀͝͠c̴̢͇͙̙̲̯̫͈̪̀̌̓̂͗̚͜͠ơ̷̬͖̙̯͇̻̱͕͐̎d̶̛̤͉̻͆̋͋̂̀e̵̦̺̹̅͆̀͛̎̎̅.̸̡̟̰͙̮̱͍̼̾͊̑̅͊̚͝