3 ms·

Unicode is variable length even if you use 32 bits, because glyphs sometimes require multiple codepoints. People sometimes write as if using more bits will remo

Unicode is variable length even if you use 32 bits, because glyphs sometimes require multiple codepoints. People sometimes write as if using more bits will remove complexity from Unicode but it doesn't really, you still need to handle multiple units at once sometimes.

Dylan16807 3y ago
Sure but that's at another level entirely. Dealing with two types of variable width is double difficult.
- asveikau 3y ago
  I disagree, if you are correctly handling Unicode you already are going through some "decode" function which parses a 32 bit quantity from utf-8 or utf-16. From there you need to handle multiple codepoints together sometimes, like for non-composed diacritics, han unification, and some emojis. This is complex regardless of whether or not you use utf-8 or utf-16, in fact I'd say it's more difficult to handle than those.
  - Dylan16807 3y ago
    We're not talking about fully correctly handling Unicode, we're talking about teaching someone the basics of an encoding. Context is critical here.
    - asveikau 3y ago
      If you want to teach someone how an encoding works, why would you not tell them that a single symbol can take multiple codepoints? It seems like you're advocating people learning incorrect information and forming their impressions of it on falsehood. Which is probably why people think utf-32 frees you from variable-length encoding.
      - Dylan16807 3y ago
        You should tell them, yes. And talk about it more at some point. But you don't have to go into much detail when today's lesson is specifically teaching UTF-8 or UTF-32. I don't know about you, but I think I could teach the latter about ten times faster. As part of a comprehensive dive into Unicode it's a minor part, but for teaching an encoding it's a significant difference.
        asveikau 3y ago
        > I think I could teach the latter about ten times faster. I've lectured computer science at the university level, and I think you could introduce all this information to a CS undergrad pretty coherently and design a lab or small assignment on it no problem. Maybe you could ask them to parse some emojis that require multiple 32-bit codepoints.
        Dylan16807 3y ago
        That sounds entirely reasonable, but you have to deal with a lot less engagement when you're teaching via blog post or similar.

Dylan16807 3y ago

Sure but that's at another level entirely. Dealing with two types of variable width is double difficult.

asveikau 3y ago
I disagree, if you are correctly handling Unicode you already are going through some "decode" function which parses a 32 bit quantity from utf-8 or utf-16. From there you need to handle multiple codepoints together sometimes, like for non-composed diacritics, han unification, and some emojis. This is complex regardless of whether or not you use utf-8 or utf-16, in fact I'd say it's more difficult to handle than those.
- Dylan16807 3y ago
  We're not talking about fully correctly handling Unicode, we're talking about teaching someone the basics of an encoding. Context is critical here.
  - asveikau 3y ago
    If you want to teach someone how an encoding works, why would you not tell them that a single symbol can take multiple codepoints? It seems like you're advocating people learning incorrect information and forming their impressions of it on falsehood. Which is probably why people think utf-32 frees you from variable-length encoding.
    - Dylan16807 3y ago
      You should tell them, yes. And talk about it more at some point. But you don't have to go into much detail when today's lesson is specifically teaching UTF-8 or UTF-32. I don't know about you, but I think I could teach the latter about ten times faster. As part of a comprehensive dive into Unicode it's a minor part, but for teaching an encoding it's a significant difference.
      - asveikau 3y ago
        > I think I could teach the latter about ten times faster. I've lectured computer science at the university level, and I think you could introduce all this information to a CS undergrad pretty coherently and design a lab or small assignment on it no problem. Maybe you could ask them to parse some emojis that require multiple 32-bit codepoints.
        Dylan16807 3y ago
        That sounds entirely reasonable, but you have to deal with a lot less engagement when you're teaching via blog post or similar.