3 ms·

> Did Unicode consider using semantic language markers? Once upon a time Unicode had inline language tags [1]. [1] https://en.wikipedia.org/wiki/Tags_(Unicode

> Did Unicode consider using semantic language markers? Once upon a time Unicode had inline language tags [1]. [1] https://en.wikipedia.org/wiki/Tags_(Unicode_block)#Legacy_use https://en.wikipedia.org/wiki/Tags_(Unicode_block)#Legacy_us... > Those could even help unify some latin alphabets and save you even more codepoints, [...] This is a common misunderstanding; they may look alike but can and often do have different rules (e.g. ordering) underneath, so they have to be disunified.

eviks 3y ago
Thanks, pity it got poorly designed and cancelled, do you know of a good document/criticism re. why that happened? The Wikipedia linked memos are process garbage docs where copyright header/footer with authors' addresses take more space than the (lack of) substance > they may look alike but can and often do have different rules (e.g. ordering) underneath, so they have to be disunified. Don't get it, you'd have different rules apply depending on the language tags, why would you need multiple codepoints? Besides, the codepoint could be "first letter of a Language alphabet" instead of "a letter that looks like A"
- lifthrasiir 3y ago
  It's not exactly "cancelled" per se, and you need to look at the correct place (which indeed takes time to learn...). At the beginning there was an Internet-Draft [1] proposing the use of invalid UTF-8 sequences, with an initial intention to be used in the Application Configuration Access Protocol (RFC 2244). Take a look at that---it is really horrible if you think about that. Nevertheless the proposal was significant because it demonstrated needs for "lightweight language tagging" and a more reasonable proposal was developed in the Unicode Technical Committee [2], which resulted in the current Tags block. Since they were meant to be used only when there is no other markup method, though, it was "born 'strongly discouraged'" from the beginning [3]. The current "deprecated" status is rather a formal confirmation of this position---[3] even clarifies that they are "indicated as 'strongly discouraged', but reserved for use with special protocols", and Unicode never removes once allocated code points, so Unicode can't do anything more about deprecated characters. Still, the intent is that you should use correct `lang="..."` attributes whenever applicable. [1] https://datatracker.ietf.org/doc/html/draft-ietf-acap-mlsf-00 https://datatracker.ietf.org/doc/html/draft-ietf-acap-mlsf-0... [2] https://www.unicode.org/L2/L1997/97255r.pdf#page=4 https://www.unicode.org/L2/L1997/97255r.pdf#page=4 [3] https://www.unicode.org/L2/L2001/01301-deprecation.txt https://www.unicode.org/L2/L2001/01301-deprecation.txt > Don't get it, you'd have different rules apply depending on the language tags, why would you need multiple codepoints? That means you need to first locate all language tags before doing anything, which is wasteful. Almost all Unicode operations are designed to function with only local contexts in comparison.

eviks 3y ago

Thanks, pity it got poorly designed and cancelled, do you know of a good document/criticism re. why that happened? The Wikipedia linked memos are process garbage docs where copyright header/footer with authors' addresses take more space than the (lack of) substance > they may look alike but can and often do have different rules (e.g. ordering) underneath, so they have to be disunified. Don't get it, you'd have different rules apply depending on the language tags, why would you need multiple codepoints? Besides, the codepoint could be "first letter of a Language alphabet" instead of "a letter that looks like A"

lifthrasiir 3y ago
It's not exactly "cancelled" per se, and you need to look at the correct place (which indeed takes time to learn...). At the beginning there was an Internet-Draft [1] proposing the use of invalid UTF-8 sequences, with an initial intention to be used in the Application Configuration Access Protocol (RFC 2244). Take a look at that---it is really horrible if you think about that. Nevertheless the proposal was significant because it demonstrated needs for "lightweight language tagging" and a more reasonable proposal was developed in the Unicode Technical Committee [2], which resulted in the current Tags block. Since they were meant to be used only when there is no other markup method, though, it was "born 'strongly discouraged'" from the beginning [3]. The current "deprecated" status is rather a formal confirmation of this position---[3] even clarifies that they are "indicated as 'strongly discouraged', but reserved for use with special protocols", and Unicode never removes once allocated code points, so Unicode can't do anything more about deprecated characters. Still, the intent is that you should use correct `lang="..."` attributes whenever applicable. [1] https://datatracker.ietf.org/doc/html/draft-ietf-acap-mlsf-00 https://datatracker.ietf.org/doc/html/draft-ietf-acap-mlsf-0... [2] https://www.unicode.org/L2/L1997/97255r.pdf#page=4 https://www.unicode.org/L2/L1997/97255r.pdf#page=4 [3] https://www.unicode.org/L2/L2001/01301-deprecation.txt https://www.unicode.org/L2/L2001/01301-deprecation.txt > Don't get it, you'd have different rules apply depending on the language tags, why would you need multiple codepoints? That means you need to first locate all language tags before doing anything, which is wasteful. Almost all Unicode operations are designed to function with only local contexts in comparison.