3 ms·

What's your use case for iterating or indexing into code points?

• ubernostrum 9y ago
  My use case is that I don't want iterating/indexing to yield the underlying byte representation, since that may end up slicing up a codepoint large enough to require multiple bytes. I want the fundamental constituent unit of my strings to be the thing that the string is conceptually made of. When I'm ready to turn it into a sequence of bytes, I'll do that myself.
  • patrec 9y ago
    But the text is not conceptually made of unicode code points. It's conceptually made of characters. Codepoints are basically ever only used for either a) low-level mucking around like normalization -- a fringe use b) a (in the general case) bad approximation to characters -- the normal use, which will produce bad results for various "atypical" cases Now, personally, I'd often rather muck around with bytes because frequently they are a more useful abstraction than codepoints and lots of in practice important text operations can be performed quite fine (and efficiently) on them, even if the text is full of non-ascii characters.