5 ms·
What is with the push to unicode? Why not ascii? It seems to give a lot less trouble, particularly wrt panagrams, normalization, etc.
by mises 8y ago
What is with the push to unicode? Why not ascii? It seems to give a lot less trouble, particularly wrt panagrams, normalization, etc.
- totoglazer 8y agoI think it might have something to do with the ~7 billion people whose native language isn’t expressible in ascii. Just maybe?
- happytoexplain 8y agoASCII is effectively just the keys on a US English keyboard. You couldn't even represent an accented 'e' or the price of British products. So even if you just wanted a character set defined as "very common characters in the English speaking world", ASCII falls miles short of even that tiny goal.
- mises 8y agoMost of the internet is based on American English. See programming for an example. For the goal of representing American English, it works fine. Why would anything more be necessary?
- function_seven 8y agoUnicode is not about the Internet. Just like ASCII or EBCDIC before it weren't about networking. The reason we use Unicode is because ASCII is very limited in its scope. It can't handle the majority of the world's languages. It can't even handle American English. No dashes, no open or close quotes, etc.
- mises 8y agoMost programming is in American english too. And most people who use computers end up picking up a decent bit anyway, according to some friends from a spanish-speaking country.
- function_seven 8y agoSo ASCII is good enough for programming, but fails at general text, unless we resign ourselves to kludges like -- instead of – and --- instead of —. And your friends from that Spanish-speaking country do UI work right? Can’t use ASCII for that. Ñ is out the window.
- mises 8y agoNo, they do c.
- happytoexplain 8y agoIf you're encoding data for a domain guaranteed to contain only US English words and US English symbols, and you're OK without the occasional non-ASCII bits in US English like diacritics in loan words, then sure. But such domains are vanishingly rare. As for your examples, "most of the internet" might still contain non-ASCII, I think, but even a little bit of the internet is still a gigantic and highly active part that's inseparable from the larger domain of "the internet". And, while the legal symbols in the syntax of a programming language might only be ASCII, programming languages must process arbitrary strings, which may be user-facing, or be in some other domain where ASCII is inadequate.
- profmonocle 8y ago> Most of the internet is based on American English. Emoji are rapidly becoming a staple of American English. (Edit: and apparently HN filters emoji out of posts!)
- kalleboo 8y agoそうですね、僕も分からない（´-`）⁉️
- NegativeLatency 8y agoGoogle Translate: "Well, I do not know"
- sevensor 8y agoSpeaking of which, how does Han Unification affect Unicode normalization. If I understand correctly, you can compose strokes into characters? Does that work?
- nabla9 8y agoI have really good news for you. UTF-8 encoding is compatible with ASCII. As long as you just use ASCII characters, they your strings are also valid UTF-8.
- 2038AD 8y agoASCII is pretty limited. For example your namesake's book Nationalökonomie could not properly by rendered in ASCII.
- 2038AD 8y ago*be
- bshipp 8y agoI'm currently processing a huge archive of files where someone was too lazy to bother using the proper encoding. out of the first 700K files I've run through I've dumped 3000 into a secondary queue because they've been so bastardized the mojibake is completely indecipherable. It continues to astonish me that programmers who spend hours arguing the relative merits of ECC RAM and database ACID implementations are so quick to destroy data by lazily defaulting to ASCII.