4 ms·

Well it ends up that if you don't want to go down the Unicode rabbit hole too far then yeah, your best bet is probably to stick with ASCII. As an example from m

Well it ends up that if you don't want to go down the Unicode rabbit hole too far then yeah, your best bet is probably to stick with ASCII. As an example from my industry though, it would be nice if I could implement user name entry and display for a high score table in a simple game and support names in common European languages without needing to handle all the edge cases of e.g. mixed left to right and right to left text, combining characters, surrogates, etc. I'm far from a Unicode or a languages expert but I'm familiar with one language with right to left non Latin characters and aware of just enough Unicode madness to know I don't know enough to handle many edge cases properly. It would be nice if a regular developer like me could support something more than plain ASCII but less than the full insanity of Unicode to accommodate at least some non English users.

saagarjha 8y ago
You're basically throwing people with "inconvenient" character sets (i.e. everything that doesn't use something strongly resembling Latin characters) under the bus. Sure, you might be able to support Spanish, French, and German, but you're basically disregarding Japanese, Chinese, and Hindi when doing so (and possibly even ASCII, since you'd trade some symbols for accents).
- mattnewport 8y ago
  That's not at all what I'm advocating. My point is that the extreme difficulty of fully supporting Unicode with all of its complex edge cases (like my examples of mixing left to right and right to left languages) means that the two most common outcomes are throwing up your hands and giving up supporting anything but English and just using ASCII or having broken, partial Unicode support. I'm suggesting it would be nice to have another option where you could provide some level of support for non English languages with something you have some hope of implementing correctly. Applications that correctly handle editing of mixed left to right and right to left text are rare for example but you could support Farsi speakers reasonably well in many applications without handling that scenario.
bradleyjg 8y ago
There’s Latin-1 which gets you full coverage on many European languages and almost complete coverage on several others (e.g. French uses œ, which it doesn’t have but French readers will be able to understand if oe is substituted). The problem with Latin-1 and other 1-byte options is that, unlike ASCII, they aren’t forward compatible with utf-8, which is the emerging de facto string exchange encoding. For a stand alone video game, maybe that doesn’t matter but for anything network enabled it can be a big issue.

saagarjha 8y ago

You're basically throwing people with "inconvenient" character sets (i.e. everything that doesn't use something strongly resembling Latin characters) under the bus. Sure, you might be able to support Spanish, French, and German, but you're basically disregarding Japanese, Chinese, and Hindi when doing so (and possibly even ASCII, since you'd trade some symbols for accents).

mattnewport 8y ago
That's not at all what I'm advocating. My point is that the extreme difficulty of fully supporting Unicode with all of its complex edge cases (like my examples of mixing left to right and right to left languages) means that the two most common outcomes are throwing up your hands and giving up supporting anything but English and just using ASCII or having broken, partial Unicode support. I'm suggesting it would be nice to have another option where you could provide some level of support for non English languages with something you have some hope of implementing correctly. Applications that correctly handle editing of mixed left to right and right to left text are rare for example but you could support Farsi speakers reasonably well in many applications without handling that scenario.

bradleyjg 8y ago

There’s Latin-1 which gets you full coverage on many European languages and almost complete coverage on several others (e.g. French uses œ, which it doesn’t have but French readers will be able to understand if oe is substituted). The problem with Latin-1 and other 1-byte options is that, unlike ASCII, they aren’t forward compatible with utf-8, which is the emerging de facto string exchange encoding. For a stand alone video game, maybe that doesn’t matter but for anything network enabled it can be a big issue.