3 ms·
>We’ve seen four different lengths so far: Number of UTF-8 code units (17 in this case) Number of UTF-16 code units (7 in this case) Number of UTF-32 code unit
by mrheosuper 1y ago
>We’ve seen four different lengths so far:
Number of UTF-8 code units (17 in this case)
Number of UTF-16 code units (7 in this case)
Number of UTF-32 code units or Unicode scalar values (5 in this case)
Number of extended grapheme clusters (1 in this case)
We would not have this problem if we all agree to return number of bytes instead.
Edit: My mistake. There would still be inconsistency between different encoding. My point is, if we all decided to report number of bytes that string used instead number of printable characters, we would not have the inconsistency between languages.
- com2kid 1y agoHow would that help? UTF-8, 16, and 32 languages would still report different numbers.
- curtisf 1y ago"number of bytes" is dependent on the text encoding. UTF-8 code units _are_ bytes, which is one of the things that makes UTF-8 very nice and why it has won
- ivanjermakov 1y agoI would say Unicode has won, but not UTF-8. UTF-16 is also widely used due to its efficiency on asian texts.
- minebreaker 1y ago> We would not have this problem if we all agree to return number of bytes instead. I don't understand. It depends on the encoding isn't it?
- charcircuit 1y ago>Number of extended grapheme clusters (1 in this case) Only if you are using a new enough version of unicode. If you were using an older version it is more than 1. As new unicode updates come out, the number of grapheme clusters a string has can change.
- baq 1y agowhen I'm reading text on a screen, I very much am not reading bytes. this is obvious when you actually think what 'text encoding' means.
- account42 1y agoYou're not reading unicode code points either though. Your computer uses bytes, you read glyphs which roughly correspond to unicode extended grapheme clusters - anything between might look like the correct solution at first but is the wrong abstraction for almost everything.
- baq 1y agoyou are right, but this just drives the point.
- jibal 1y ago> if we all decided to report number of bytes that string used instead number of printable characters But that isn't the same across all languages, or even across all implementations of the same language.