2 ms·
I don't want to trot out my whole "Unicode Rant" here, but I'll say that each human language and it's digital representation should be an object of study in its
by carapace 7y ago
I don't want to trot out my whole "Unicode Rant" here, but I'll say that each human language and it's digital representation should be an object of study in its own right.
Having a standard mapping from numbers to little graphical ideas is great. Trying to digitalize all the world's languages from a conceptual basis of "they're like English but with quirks" seems to me to be a dodge.
In other words, this article describes the kind of thing that I think should be what the Unicode people do: map out and specify datastructures and algorithms, libraries, for each language.
- kbenson 7y agoWe're at the point that it might make sense to map languages to a digital format rather than map an extensible digital format to myriad different languages. I mean, it will never happen, but if we had a much simpler format than Unicode that just supported a number of characters, but no combining characters, no accents (and perhaps even no case), and let each language decide how they wanted to represent themselves within that medium (are accents important enough to require a separate character, or is leaving it off or just using two characters next to each other sufficient?). That isn't to say we wouldn't want accurate representations of languages (as Unicode makes an attempt to do), but there's a difference between trying to represent a language, and trying to represent the data within a language. In some cases, the former is important, in others, the latter vastly outweighs the former, and making that easily accessible, categorized, compared, etc has value. I think it's only even conceivable at this point because of how much communication is done online, and in abridged form. We've already adopted pidgin languages for ad-hoc communication (as in IMs, SMS, and tweets), so I don't think people would find this that hard to grapple with at this point.
- carapace 7y agoWell the deeper problem may be that both mapping directions are impossible. No one has proved that natural languages can be fully represented in a Turing machine, eh? In other words, ASCII is not an encoding of English. It's a fluke (or perhaps a prerequisite) that a file of bytes can represent as much of English as it does.