3 ms·
Is there a reason unicode doesn't have such a builtin lang tag? Similar to right-to-left and left-to-right it could help in displaying differently otherwise ide
by gpderetta 5y ago
Is there a reason unicode doesn't have such a builtin lang tag? Similar to right-to-left and left-to-right it could help in displaying differently otherwise identical text.
It could be stored in-band with the text with little changes to existing systems. The only change would be on the presentation layer, and if the tag were to be a non printable character, it would be backward compatible. An input device could implicitly tag input texts depending on the default lang.
You need some form of sanitization, but you need it for right-to-left and left-to-right already.
- arp242 5y agoActually, this exists already – there's U+E0001 (LANGUAGE TAG) and U+E007 (CANCEL TAG) and you can put a language code between those, e.g. "\uE0001ja-JP\ue007". Its use is also deprecated and discouraged. According to [1] it's often not needed, and [2] states that it puts a lot of burden on implementations and best done at a higher level such as HTTP, HTML, etc. I have no opinion on [1] as I don't speak these languages, but I do know I really hate working with these "invisible characters" in Unicode both as a user and developer. Copy an extra invisible LTR thingy or display variant codepoint and stuff can look and behave different, and it may not at all be obvious what the hell is going on (especially for those without a technical background). [1]: https://www.unicode.org/versions/Unicode14.0.0/ch05.pdf#G11552 https://www.unicode.org/versions/Unicode14.0.0/ch05.pdf#G115... [2]: https://www.unicode.org/versions/Unicode14.0.0/ch23.pdf#G30110 https://www.unicode.org/versions/Unicode14.0.0/ch23.pdf#G301...
- lifthrasiir 5y ago> there's U+E0001 (LANGUAGE TAG) and U+E007 (CANCEL TAG) and you can put a language code between those, e.g. "\uE0001ja-JP\ue007". "ja-JP" part is also written in tag characters, so it's actually E0001 E006A E0061 E002D E004A E0050 E007F and doesn't render even in unsupported environments.
- arp242 5y agoOh right; I never actually used it, I just know it exists.
- layer8 5y ago> Is there a reason unicode doesn't have such a builtin lang tag? It does, but they aren’t widely supported and their use is not recommended, see http://unicode.org/faq/languagetagging.html http://unicode.org/faq/languagetagging.html and https://datatracker.ietf.org/doc/html/rfc6082 https://datatracker.ietf.org/doc/html/rfc6082. The recommendation is to use markup languages instead to carry that information. The reasoning is that anything related to styling is out-of-scope for Unicode (except where needed for round-trip compatibility with other character sets), or else people will also want tags for bold, italic, monospace, or (expressed semantically) for emphasis, code, etc. That’s what markup languages like HTML are for.