3 ms·
string.length() still doesn't make sense in Python 3 There's nothing "simple" about "list of characters" in Unicode. Discussed at length at: https://news.ycom
by papln 7y ago
string.length() still doesn't make sense in Python 3
There's nothing "simple" about "list of characters" in Unicode.
Discussed at length at: https://news.ycombinator.com/item?id=18154667 https://news.ycombinator.com/item?id=18154667
Here's how a Unicode export with 20 years of experience in Unicode explained it:
https://blog.golang.org/strings https://blog.golang.org/strings
> Some people think Go strings are always UTF-8, but they are not: only string literals are UTF-8. As we showed in the previous section, string values can contain arbitrary bytes; as we showed in this one, string literals always contain UTF-8 text as long as they have no byte-level escapes.
> In fact, the definition of "character" is ambiguous and it would be a mistake to try to resolve the ambiguity by defining that strings are made of characters.
> [Exercise: Put an invalid UTF-8 byte sequence into the string. (How?) What happens to the iterations of the loop?]
If you aren't in Unicode, then "character" is the same as "byte" so splitting the types gets you nothing of vlaue.
Assuming you are not implementing a low-level Unicode library, what's your use case for "list of Unicode characters" ?
- microcolonel 7y agoEven in Unicode-aware applications, it is generally not that useful to segment just by codepoint, you more often really want to segment by extended grapheme cluster.