4 ms·
Short version: file paths are an OS-specific construct, and not all OSes have paths that can be round-tripped through Unicode. The first bad option is to say "
by jmillikin 6y ago
Short version: file paths are an OS-specific construct, and not all OSes have paths that can be round-tripped through Unicode.
The first bad option is to say "paths are bytes", which is true on traditional POSIX platforms. Then you run your code on macOS and discover it's processing your bytes as UTF-8 and applying Unicode normalization. After fixing that you try porting to Windows and get to learn about `wchar_t`.
The other bad option is to say "paths are (Unicode) strings", which works fine on Windows and macOS. It works 90% of the time on POSIX, but you'll eventually find users with non-UTF8 file paths and then you're in trouble.
My first hand experience with this going wrong is GHC, which started off with raw bytes, then moved to strings using a bad conversion function, then they tried to work around the issue by placing non-UTF8 bytes into a Unicode reserved area.
Java also has problems because it represents paths as strings and tries to use the libc locale for decoding. https://github.com/bazelbuild/bazel/pull/10111 https://github.com/bazelbuild/bazel/pull/10111 is an example of how this breaks software in ways that are difficult to work around.
- migueloller 6y agoThanks for the response!