4 ms·
> Nobody wants to go back to a world with hundreds of relevant and thousands of legacy encodings. Japan does (assuming agreeing a unicode-like encoding that ac
by lmm 3y ago
> Nobody wants to go back to a world with hundreds of relevant and thousands of legacy encodings.
Japan does (assuming agreeing a unicode-like encoding that actually worked is off the table), for the simple reason that practical support for Japanese in "international" applications has regressed since those days.
> Obviously much better when nobody can read anything than forcing some people to read texts where there's the occasional odd ('non-native' if you exaggerate a bit) font style choice.
https://wiki.c2.com/?AlmostCorrect https://wiki.c2.com/?AlmostCorrect . Failing obviously and universally is better than failing subtly in edge cases.
> What you can do today is (taking HTML+CSS as the obvious choice) is proper font choice, `@font-face` declarations, proper downloadable fonts, proper configuration of OpenType font feature flags, if necessary on the basis of Unicode code point ranges. (There's also Unicode variation selectors, bit I've never worked with those so can't say how well those work.)
You can do a lot in HTML because HTML has a lot of specific support - something that would likely also be true in a world without unicode. (In the unicode world we have only one encoding per HTML file, but there's no reason a file format couldn't support multiple encodings in the same way as supporting multiple language spans etc.).
> It's not like the pre-Unicode chaos of mutually incompatible encodings did anything to help you with any of these points. Rather, it locked you into a rather small (when compared to Unicode) set of codepoints, and you typically only could do one setting per document.
The old system made the easy things easy. In those days effectively any program that could display French or Swedish documents could also display Japanese documents.
The unicode world makes some things that were previously hard easier, such as mixing languages (as long as the lanugages you want to mix aren't Japanese), but your file format has to go beyond "plain" text for the most basic functionality of properly displaying monolingual Japanese documents.
In the old days if you didn't think about languages at all your program would be broken in every non-English language. Now it's broken only in Japan. That's an improvement for most places, but it's a huge regression in Japan, because previously most programs made some effort to deal with internationalisation issues and now they don't, they just say "it's unicode so we don't have to think about it".
- DemocracyFTW2 3y agoI think almost everything you said relies on seeing the old ways through rosy glasses and massively underestimating the daily problems that the multitude (technically, gazillions) of encoding standards and standards on how to encode encoding choices brought to the many people who already worked regularly with computers in the early 90s and before that. I'll just pick a single point I disagree with, and then comment on the one point that to me has true merit. So you write, You can do a lot in HTML because HTML has a lot of specific support - something that would likely also be true in a world without unicode. (In the unicode world we have only one encoding per HTML file, but there's no reason a file format couldn't support multiple encodings in the same way as supporting multiple language spans etc.). Basically you reject a format—HTML+Unicode—that has seen tremendous global acceptance, that has brought us 'rich text' (to quote Microsoft), near-complete universal language support for hundreds of previously entirely unencoded writing systems, ability to display ruby text (i.e. small annotations, parallel version of text; btw. not limited to Japanese at all) and want to replace that with 'a file format [that] support[s] multiple encodings'. I agree that it would sometimes be nice (i.e. nice-to-have) inline encoding switches in HTML. You could totally write a JS method to do exactly that, right now, so go ahead! But to replace HTML+Unicode with 'some unspecified file formats' that 'could have' multiple encodings in a single file? Doesn't sound like I want to go down that particular cul de sac. OK now on to that other point: you write your file format has to go beyond "plain" text for the most basic functionality of properly displaying [some features for some linguistic situations], if I may be so bold and generalize to avoid that fixation on "boo it's Japanese again and only Japanese that has been put at a disadvantage". So where I agree with you is that historically the Unicode consortium was too reluctant to acknowledge responsibility for some features of written text that should be expressible in the very text, not in a 'side channel' (e.g. in HTML tags). This would IMO include markers for start and end of stretches of a language, so you could write "good day, {lang=fr}Messieur{/lang}" where the curlies symbolize where Unicode control character sequences (not unlike those already used for flags) should appear. There's more stuff like that like e.g. indicating position of text relative to other text; we sort of have it in the form of CJK Ideographic Description Characters and Control Character for Ancient Egyptian, but nothing for the general purpose.
- lmm 3y ago> I think almost everything you said relies on seeing the old ways through rosy glasses and massively underestimating the daily problems that the multitude (technically, gazillions) of encoding standards and standards on how to encode encoding choices brought to the many people who already worked regularly with computers in the early 90s and before that. It's not rosy glasses lol, I have to deal with encoding differences every day. I'd like nothing more than to be able to forget about them if there was something like unicode that actually worked. Instead I watch support for the language I'm using degrade day by day because everyone high-fives each other that they've got unicode now so they've solved all the problems (and they have! As long as you're not using Japanese), and then they add insult to injury by trying to tell me it's better.
- DemocracyFTW2 3y agoFailing obviously and universally is better than failing subtly in edge cases Sure when you have unlimited time and unlimited resources why accept anything that is even a hair's width not quite like what you intended—perfection? Meanwhile we have Unicode with all its flaws. Sick!