3 ms·

That the beauty of it with python: - if you want to treat paths like unicode strings, you can. Which is great for simple scripts where you don't want to deal w

That the beauty of it with python: - if you want to treat paths like unicode strings, you can. Which is great for simple scripts where you don't want to deal with complexity. And 99% of the time, it's enough with modern OSes. - if you want to threat path as bags of raw bytes, you can. Which is necessary to transparently copy and do not evaluate, as you said, for covering edge cases. - if you need to actually deal with those as strings but don't want to loose data for edge cases, so a mix of the 2 above, you can use surrogate escapes

mjevans 5y ago
Where does that filepath come from? A config file; are you going to do your text processing, and interaction, with other modules in byte[] arrays in Python 3+? Python 2's unicode model was _closer_ to correct, the trivial coercion between byte[] and Unicode. Conversion also shouldn't imply, force, or check Validation nor Normalization. Labeling a bytestream with an Encoding and validating / normalizing that encoding should be options. Operations on bytestreams with encoding related attributes should set them to either 'unknown' result or to a proper output type if they're aware the manipulations will still yield a valid encoding. Normalization is more complex, since Unicode strings can be normalized in different ways, then combined, and still be a valid string but no longer uniformly normalized.