4 ms·
BTW, I agree that you want to differentiate between bytes and text, but the underlying representation should be utf-8 bytes, not some constant time indexable ut
by patrec 9y ago
BTW, I agree that you want to differentiate between bytes and text, but the underlying representation should be utf-8 bytes, not some constant time indexable utf-32, which is a semantics and performance disaster. It's much better to mess up in not having a clear distinction between bytes and text (annoying, but mostly easy to work around) then to mess up by having constant-time indexable utf-32 as python3 does (not at all easy to work around and even the bits that are easy to work around, like pretending stdin or stdout and sys.argv are unicode on unix are still super-annoying).
- masklinn 9y ago> BTW, I agree that you want to differentiate between bytes and text, but the underlying representation should be utf-8 bytes Sure, but for most languages older than a few years this ship has sadly long sailed and few would be willing to switch to O(n) indexing. > It's much better to mess up in not having a clear distinction between bytes and text (annoying, but mostly easy to work around) Except when it's not and the packages you're trying to use simply do not work and fuck up mightily with non-ascii data. > then to mess up by having constant-time indexable utf-32 as python3 does It's funny that just above you praise Erlang which does exactly that. Meanwhile Python 3.3 introduced a flexible string representation so depending on the content of the string the internal string encoding may be ISO-8859-1, UCS2 or UCS4 (and yes sadly only these as the designers refused to break the constant-time indexing guarantee).