4 ms·
I used strings to mean both. Byte strings can be sliced and indexed no problems because a byte as such is something you may actually want to deal with. Slicing
by DasIch 11y ago
I used strings to mean both. Byte strings can be sliced and indexed no problems because a byte as such is something you may actually want to deal with.
Slicing or indexing into unicode strings is a problem because it's not clear what unicode strings are strings of. You can look at unicode strings from different perspectives and see a sequence of codepoints or a sequence of characters, both can be reasonable depending on what you want to do. Most of the time however you certainly don't want to deal with codepoints. Python however only gives you a codepoint-level perspective.
Guessing encodings when opening files is a problem precisely because - as you mentioned - the caller should specify the encoding, not just sometimes but always. Guessing an encoding based on the locale or the content of the file should be the exception and something the caller does explicitly.
- aidos 11y agoIt slices by codepoints? That's just silly, so we've gone through this whole unicode everywhere process so we can stop thinking about the underlying implementation details but the api forces you to have to deal with them anyway. Fortunately it's not something I deal with often but thanks for the info, will stop me getting caught out later.