4 ms·

I believe the strings are still encoded internally as byte arrays however, it's just that when you pull out a character it can be multiple bytes (emojis for exa

• dhosek 6y ago
  Yes, that's what I understand about how it works. There's a lot to be said against the idea of dealing with Unicode at the codepoint level in most applications though. I'm writing code that would allow commands to be either \ + non-letter character or \ + one or more letters (a la TeX). So that means that I would allow commands to include \bird, \pták, \طائر and \鳥, but what happens if the á in \pták is input as ´+ a rather than á? Is that supposed to be the same code? Or perhaps the user has \Spi¨nalTap (and there code editor has a type compositor that's willing to put the umlaut on the n?) Some of this can be dealt with through Unicode normalization, although there's also the question of whether, e.g., the ohm and angstrom symbols should be treated as symbols or letters if they're input at their symbolic code points rather than as a Greek or Latin letters. Would \+white man shrugging be a valid \+non-letter command since it's multiple code points? It's amazing how much a "simple" specification gets complicated when you start to look at all the ways that Unicode can complicate matters.