4 ms·

Italic and bold isolate characters for Unicode

1 points by luu 3y ago

WantonQuantum 3y ago
Pro: Viewing text with bold and italics is supported by anything that supports (up to date) Unicode. No other formatting needed. But really when was the last time you looked at some text in a context that didn’t include some formatting? A text editor. That’s about it. An IDE. Con: Everything in the universe that can edit text now has to handle new ways to encode bold and italics and choose between the character encoding or the document encoding for saving it. How is that choice made? On the whole, seems like we should keep the character encoding in Unicode and the formatting in the enclosing document format. Besides, we’d need to add underline, strike through, etc.
Someone 3y ago
O, the ‘fun’ this would be once you layer another formatting library such as markdown or html on top of it, or combine it with Unicode’s mathematical alphanumerical symbols (https://www.unicode.org/charts/PDF/U1D400.pdf https://www.unicode.org/charts/PDF/U1D400.pdf) Luckily, I don’t think the Unicode Consortium will ever accept something like this. Unicode has lots of warts, but adding completely new ones isn’t popular with them.
DemocracyFTW2 3y ago
I oppose this proposal not because it brings formatting codes to Unicode, but because it is too parsimonious, not extensible, and too Latin-centric. It is too parsimonious because it only discusses bold and italic characters, but these are just two out of a number of character formatting one will want to have in a given text. It is not extensible because it uses four special codepoints to implement just two formatting modes; it sure is extensible in the way that we still have maybe ~500.000 free codepoints in Unicode so not in danger of running out of them any time soon, but the proposal is not extensible in the way that, say, HTML is, a markup where I can just make up [sic] any new tag `<foo-bar>` and browsers will swallow it (and process it correctly when given some CSS), all without changing the basic assumptions of HTML markup (that only `<`, `>`, and `&` are characters with special meanings, that `<tag>` is closed by a symmetric `</tag>` and so on). Lastly, the proposal is Latin-centric in proposing a dedicated syntax for bold and italic, something that does not apply to all scripts. For example, Sinographs (Chinese characters, Kanji) do not have italics per se, and they do not lend themselves well to algorithmic emboldening. Instead what you should do is choose another font or the results will (and often enough do) look gross and hard-to-read. Instead, I argue, what we need is a generic mechanism to mark up relative positioning and 'roles', as it were, much as you'd do in semantic HTML/CSS, but with a focus on typesetting. Let me exemplify this with diacritics. Unicode wanted to have roundtrip compatibility with a (big) number of existing encodings which is why they chose to encode things like `²` and `ä` as single codepoints. At the same time, it was recognized that the number of characters that one would want to adorn with a `¨` (diaresis) was practically unlimited, and given that it was desirable to enable exotic combinations like, say, `ç̈` (Latin small letter c with cedilla and diaresis), it was out of the question to do so by assigning separate codepoints to all possible combinations. Instead, the Consortium went for the next best solution and introduced non-combining and combing versions of diacritics. I'd say this was a mistake because while it does enable a huge number of accessible possibilities it still is not generic enough to support truly ad-hoc combinations. For one thing, you need the Standard to adopt one code point for 'combining placement of x above' and one for 'x below' so now you have three codepoints for each diacritic. Instead, it would have been better to introduce a generic markup that indicates formatting, plus commands to indicate 'place this on top of', 'place this below', much like the HTML tags `<sup>` and `<sub>` do. Once you have that, you can use any Unicode glyph and put it on top of another in plain text markup. There are many more of such desirable formatting codes and I have here skimmed over many details, e.g. the question of how to size stuff (I might want to stack an `x` on top of a `y` in full size, or I might want to have a tiny `x` on top of a full-sized `y`, &cpp). And wouldn't you know it—this is exactly what they have done with Egyptian hieroglyphic formatting codes[1] in v15. One has to make friends with the fact that Unicode does not always follow through with its principles, that some theoretically stated principles did not work out to be practical, that some writing systems although superficially similar are using radically different ways to encode stuff and so on: for example, we now have dedicated codepoints for the ASCII characters in bold, italic, and Fraktur, which violates early principled statements of the Standard; there are hundreds upon thousands of CJK characters where some variants are separately encoded while others are not; there's the case of Coptic which used to be unified with Greek but was later given a space of its own; there's Tibetan where, apparently due to some personal rivalry among proponents we now have an inane encoding where all the consonants that can participate in consonant stacks / ligatures have now codepoints of their own, violating established encoding practices valid for all other scripts of the Indian type, and so on. It has also been criticized that there are 11.000 codepoints for some (not all) possible Hangŭl quadrats while there are also combining and non-combining Jamo (letters) so there's considerable overlap. But some mistakes have been corrected and more will have to be corrected. The introduction of formatting codes for Egyptian hieroglyphs tells me that, in principle, my argumentation is not altogether unaligned with the present stance of the Consortium at this point. But introducing special magic brackets to do that one favorite trick of yours, that's hopefully not going to fly. [1](https://en.wikipedia.org/wiki/Egyptian_Hieroglyph_Format_Controls https://en.wikipedia.org/wiki/Egyptian_Hieroglyph_Format_Con...)

1 points by luu 3y ago

Pro: Viewing text with bold and italics is supported by anything that supports (up to date) Unicode. No other formatting needed. But really when was the last time you looked at some text in a context that didn’t include some formatting? A text editor. That’s about it. An IDE. Con: Everything in the universe that can edit text now has to handle new ways to encode bold and italics and choose between the character encoding or the document encoding for saving it. How is that choice made? On the whole, seems like we should keep the character encoding in Unicode and the formatting in the enclosing document format. Besides, we’d need to add underline, strike through, etc.

Someone 3y ago

O, the ‘fun’ this would be once you layer another formatting library such as markdown or html on top of it, or combine it with Unicode’s mathematical alphanumerical symbols (https://www.unicode.org/charts/PDF/U1D400.pdf https://www.unicode.org/charts/PDF/U1D400.pdf) Luckily, I don’t think the Unicode Consortium will ever accept something like this. Unicode has lots of warts, but adding completely new ones isn’t popular with them.

DemocracyFTW2 3y ago

I oppose this proposal not because it brings formatting codes to Unicode, but because it is too parsimonious, not extensible, and too Latin-centric. It is too parsimonious because it only discusses bold and italic characters, but these are just two out of a number of character formatting one will want to have in a given text. It is not extensible because it uses four special codepoints to implement just two formatting modes; it sure is extensible in the way that we still have maybe ~500.000 free codepoints in Unicode so not in danger of running out of them any time soon, but the proposal is not extensible in the way that, say, HTML is, a markup where I can just make up [sic] any new tag `<foo-bar>` and browsers will swallow it (and process it correctly when given some CSS), all without changing the basic assumptions of HTML markup (that only `<`, `>`, and `&` are characters with special meanings, that `<tag>` is closed by a symmetric `</tag>` and so on). Lastly, the proposal is Latin-centric in proposing a dedicated syntax for bold and italic, something that does not apply to all scripts. For example, Sinographs (Chinese characters, Kanji) do not have italics per se, and they do not lend themselves well to algorithmic emboldening. Instead what you should do is choose another font or the results will (and often enough do) look gross and hard-to-read. Instead, I argue, what we need is a generic mechanism to mark up relative positioning and 'roles', as it were, much as you'd do in semantic HTML/CSS, but with a focus on typesetting. Let me exemplify this with diacritics. Unicode wanted to have roundtrip compatibility with a (big) number of existing encodings which is why they chose to encode things like `²` and `ä` as single codepoints. At the same time, it was recognized that the number of characters that one would want to adorn with a `¨` (diaresis) was practically unlimited, and given that it was desirable to enable exotic combinations like, say, `ç̈` (Latin small letter c with cedilla and diaresis), it was out of the question to do so by assigning separate codepoints to all possible combinations. Instead, the Consortium went for the next best solution and introduced non-combining and combing versions of diacritics. I'd say this was a mistake because while it does enable a huge number of accessible possibilities it still is not generic enough to support truly ad-hoc combinations. For one thing, you need the Standard to adopt one code point for 'combining placement of x above' and one for 'x below' so now you have three codepoints for each diacritic. Instead, it would have been better to introduce a generic markup that indicates formatting, plus commands to indicate 'place this on top of', 'place this below', much like the HTML tags `<sup>` and `<sub>` do. Once you have that, you can use any Unicode glyph and put it on top of another in plain text markup. There are many more of such desirable formatting codes and I have here skimmed over many details, e.g. the question of how to size stuff (I might want to stack an `x` on top of a `y` in full size, or I might want to have a tiny `x` on top of a full-sized `y`, &cpp). And wouldn't you know it—this is exactly what they have done with Egyptian hieroglyphic formatting codes[1] in v15. One has to make friends with the fact that Unicode does not always follow through with its principles, that some theoretically stated principles did not work out to be practical, that some writing systems although superficially similar are using radically different ways to encode stuff and so on: for example, we now have dedicated codepoints for the ASCII characters in bold, italic, and Fraktur, which violates early principled statements of the Standard; there are hundreds upon thousands of CJK characters where some variants are separately encoded while others are not; there's the case of Coptic which used to be unified with Greek but was later given a space of its own; there's Tibetan where, apparently due to some personal rivalry among proponents we now have an inane encoding where all the consonants that can participate in consonant stacks / ligatures have now codepoints of their own, violating established encoding practices valid for all other scripts of the Indian type, and so on. It has also been criticized that there are 11.000 codepoints for some (not all) possible Hangŭl quadrats while there are also combining and non-combining Jamo (letters) so there's considerable overlap. But some mistakes have been corrected and more will have to be corrected. The introduction of formatting codes for Egyptian hieroglyphs tells me that, in principle, my argumentation is not altogether unaligned with the present stance of the Consortium at this point. But introducing special magic brackets to do that one favorite trick of yours, that's hopefully not going to fly. [1](https://en.wikipedia.org/wiki/Egyptian_Hieroglyph_Format_Controls https://en.wikipedia.org/wiki/Egyptian_Hieroglyph_Format_Con...)