3 ms·

> Everywhere[1] but in Windows UTF-8 is being used. Further popular counterexamples besides the NT kernel (cf. [2]): - Java - JavaScript - .NET - Qt - Joi

> Everywhere[1] but in Windows UTF-8 is being used. Further popular counterexamples besides the NT kernel (cf. [2]): - Java - JavaScript - .NET - Qt - Joilet file system (for CD-ROMs) [1] http://utf8everywhere.org/ http://utf8everywhere.org/ [2] https://en.wikipedia.org/w/index.php?title=UTF-16&oldid=909417171#Usage https://en.wikipedia.org/w/index.php?title=UTF-16&oldid=9094...

new_realist 7y ago
All decisions made 20 years ago. The future is UTF-8.
- naikrovek 7y ago
  honestly the future is probably UTF-32. just make all code points 32-bits and use the 11 bits that Unicode says it will never use for flags so things like formatting can be encoded per character.
  - targonca 7y ago
    That's... a very bad idea. First of all, there's graphemes are inherently not one-to-one with code points, e.g. Á = A + `. There's simply no Unicode encoding that will let you safely index into an array without paying attention to the meaning of the underlying codepoints. (and no, using NFC won't solve this either, because there are combinations for which there's no composed equivalent) Secondly, general formatting info won't fit into 11 bits (italic, bold, underline, strikethrough - that's already 4 bits, and we haven't talked about color, font weights other than bold, etc.), so why bother baking in a limited, intentionally gimped version into your character encoding?
    - naikrovek 7y ago
      It doesn't have to be formatting... It is not a "very bad idea" it is "an idea you do not like." Those are different things. The way you're describing UTF-32, it can't work at all, and it definitely does. Trying to save space by using UTF-8 over UTF-32 seems like a very small gain to me, is all. UTF-32 is simpler, for text created in that encoding.
      - targonca 7y ago
        There are tons of resources online about why UTF-32 doesn't make sense. I'm not gonna repeat them. Do your own research. https://news.ycombinator.com/item?id=8195827 https://news.ycombinator.com/item?id=8195827 https://softwareengineering.stackexchange.com/questions/236194/does-it-make-sense-to-choose-utf-32-based-on-concern-that-some-basic-rule-will https://softwareengineering.stackexchange.com/questions/2361... https://en.wikipedia.org/wiki/UTF-32#Analysis https://en.wikipedia.org/wiki/UTF-32#Analysis http://utf8everywhere.org/#myths http://utf8everywhere.org/#myths
- wolfgke 7y ago
  > All decisions made 20 years ago. The future is UTF-8. As you can see in the examples, they are all (except perhaps the Joilet file system) still very actively used. I see no reason that they will disappear soon. Also the decision to use UTF-8 under GNU/Linux is a decision that was similarly made about 20 years ago.

new_realist 7y ago

All decisions made 20 years ago. The future is UTF-8.

naikrovek 7y ago
honestly the future is probably UTF-32. just make all code points 32-bits and use the 11 bits that Unicode says it will never use for flags so things like formatting can be encoded per character.
- targonca 7y ago
  That's... a very bad idea. First of all, there's graphemes are inherently not one-to-one with code points, e.g. Á = A + `. There's simply no Unicode encoding that will let you safely index into an array without paying attention to the meaning of the underlying codepoints. (and no, using NFC won't solve this either, because there are combinations for which there's no composed equivalent) Secondly, general formatting info won't fit into 11 bits (italic, bold, underline, strikethrough - that's already 4 bits, and we haven't talked about color, font weights other than bold, etc.), so why bother baking in a limited, intentionally gimped version into your character encoding?
  - naikrovek 7y ago
    It doesn't have to be formatting... It is not a "very bad idea" it is "an idea you do not like." Those are different things. The way you're describing UTF-32, it can't work at all, and it definitely does. Trying to save space by using UTF-8 over UTF-32 seems like a very small gain to me, is all. UTF-32 is simpler, for text created in that encoding.
    - targonca 7y ago
      There are tons of resources online about why UTF-32 doesn't make sense. I'm not gonna repeat them. Do your own research. https://news.ycombinator.com/item?id=8195827 https://news.ycombinator.com/item?id=8195827 https://softwareengineering.stackexchange.com/questions/236194/does-it-make-sense-to-choose-utf-32-based-on-concern-that-some-basic-rule-will https://softwareengineering.stackexchange.com/questions/2361... https://en.wikipedia.org/wiki/UTF-32#Analysis https://en.wikipedia.org/wiki/UTF-32#Analysis http://utf8everywhere.org/#myths http://utf8everywhere.org/#myths
wolfgke 7y ago
> All decisions made 20 years ago. The future is UTF-8. As you can see in the examples, they are all (except perhaps the Joilet file system) still very actively used. I see no reason that they will disappear soon. Also the decision to use UTF-8 under GNU/Linux is a decision that was similarly made about 20 years ago.