3 ms·
This comes up because many implementations of "UTF-16" are in fact naive, and just operate within the mentality of UCS-2, where everything is just a 16-bit numb
by ucarion 8y ago
This comes up because many implementations of "UTF-16" are in fact naive, and just operate within the mentality of UCS-2, where everything is just a 16-bit number.
This is relevant within this discussion because we were talking about filesystems, and Windows stores file names as an arbitrary array of 16-bit numbers, but UTF-16 has additional requirements that Windows file names do not satisfy.
When I say UCS-2, I just mean pretending that UTF-16 is short for `[]uint16`, and not worrying about surrogate pairs.
- jrochkind1 8y agoI think we can all agree that you should never be applying any normalization to bytes representing a file system path, unless you are doing it according to specifications for that particular file system+OS combo. file system paths are tricky with non-ascii, that is for sure! Especially if you are trying to be agnostic to OS/filesystem. They are an edge case.