> The whole point of Unicode was to fix it all. The whole point of Unicode was to provide a vendor-approved universal character set and encoding. Something ISO

> The whole point of Unicode was to fix it all. The whole point of Unicode was to provide a vendor-approved universal character set and encoding. Something ISO/IEC 2022 and the original draft of ISO/IEC 10646 (which was subsequently harmonized with Unicode) couldn't do. As I'll reiterate below, it is impossible to fix "mistakes" of the past because the human text is complex and these "mistakes" are a part of that. Unicode instead had more managable (and eventually successful) goal of maintaining the compatibility with existing character sets so that it can overtake them. > There is no reason to support both combining chars and unique code points for the same glyph. Unicode should pick one and dump the other. Your second claim that combining characters can replace unique code points doesn't make sense at all. If you have "combining" characters then you also need base characters to be combined. If these base characters can be used by its own then it is no different from the current Unicode. If these base characters can't be used by its own then some characters might have two code points assigned, one combinable and one not. That would be more complex than today and can cause issues in multiple scripts. I presume that you didn't think about that second claim and only thought about the first claim that unique code points can replace combining characters. This might be possible if you disregard several important scripts, and my native Korean is one of them. Pre-Unicode Korean and Hangul support in computers was always lousy and there had been three major approaches. The first is to PRECOMPOSE every common syllable. The second is to algorithmically COMPOSE consonants and vowels into a syllable block by having byte patterns calculated from jamos themselves. The third is similar to the second but RECOGNIZEs a row of composable jamos, so the byte length of each character is variable. All three approaches have been used multiple times in the history. By following your logic Hangul should be always encoded using PRECOMPOSE or COMPOSE. After all that's how Han characters (aka "CJK" "ideographs") are encoded, so it should be workable right? And yet we are still adding a slew of additional Han characters to Unicode every two or three year. Unicode 1.0 indeed had used PRECOMPOSE, but they had to switch to COMPOSE by 2.0 because additional characters kept being added. They are characters people do use, just less frequently. The adoption of COMPOSE made Hangul one of the biggest scripts in Unicode, encompassing 11,172 characters (and in part triggering the introduction of UTF-16). So we should be fine as is, right? No, because there are another beast called archaic Hangul used before the 1933 standardization. They are about as frequent as least used modern Hangul characters, but some of them are definitely in use. And here's a catch: if you implement archaic Hangul in Unicode using COMPOSE, there would be at least 1,638,750 characters of them [1]. I'm sure you are definitely not okay with that. Back then the most popular Korean word processor, nowadays called the Hangul Office, supported archaic Hangul natively. They had their own 16-bit encoding which used COMPOSE throughout modern Hangul, but their archaic Hangul encoding was very much mixed [2]. It was a combination of all three approaches above: some frequent archaic jamos are implemented just like modern Hangul using COMPOSE, remaining frequent archaic syllables got their own code a la PRECOMPOSE, and other archaic jamos are RECOGNIZEd as a character if consecutive. All of this mess because of the limited size of their encoding. And yet it was the best possible before Unicode. The modern Unicode encoding uses both COMPOSE (modern syllables only) and RECOGNIZE, and sequences in both approaches are considered equivalent. And that equivalence is not something made up just for Korean, the same approach is valid and used for many other scripts, so if you implement the Unicode normalization you've got most things right for archaic Korean. > This is what makes it crazy and nobody implements it right, or even knows what "right" is. You may want to believe that there is the "right" way, but there isn't. Unicode gives you a standardized set of algorithms and data for what they can (equivalence, normalization, collation, ...). Unicode is very fine for what they support, but it can't help you if something is out of their scope, including fonts (you were conflating this a lot, weren't you). For those things you don't have a single universal answer and your answer may change according to which languages, scripts or locales you support and how much your users can tolerate, among others. > People use this to embed secret messages in innocuous looking text files that print identically. There's no round trip Unicode->paper->Unicode. There is no round trip between any character encoding and paper. Assume that your text is monospaced and you see something like this: This line definitely doesn't have a space after this: Are you sure that there is no space after the colon? Even if we restrict ourselves to visual characters, you can do much of steganography without Unicode (font changes, whitespace counts, keming, intentional typos, alternative expressions...). You are making your own problem up. [1] https://charset.fandom.com/ko/wiki/%EC%9C%A0%EB%8B%88%EC%BD%94%EB%93%9C_%EC%98%9B%ED%95%9C%EA%B8%80_1638750%EC%9E%90 https://charset.fandom.com/ko/wiki/%EC%9C%A0%EB%8B%88%EC%BD%... [2] https://charset.fandom.com/ko/wiki/%ED%95%9C%EC%BB%B4_2%EB%B0%94%EC%9D%B4%ED%8A%B8_%EC%BD%94%EB%93%9C/%ED%95%9C%EA%B8%80 https://charset.fandom.com/ko/wiki/%ED%95%9C%EC%BB%B4_2%EB%B...