4 ms·
There's WTF-8 to convert broken UTF-16 into something UTF-8ish. https://simonsapin.github.io/wtf-8/ https://simonsapin.github.io/wtf-8/ I kind of wonder if pat
by legulere 10y ago
There's WTF-8 to convert broken UTF-16 into something UTF-8ish.
https://simonsapin.github.io/wtf-8/ https://simonsapin.github.io/wtf-8/
I kind of wonder if paths not being allowed to contain NUL or '/' was one reason why for codepoints that are represented through more than one byte in UTF-8 (-> all non ASCII codepoints) all bytes have the most significant bit set to 1 (https://en.wikipedia.org/wiki/UTF-8#Description https://en.wikipedia.org/wiki/UTF-8#Description) This makes it impossible to have multi-byte to contain valid ascii chars like `/`.
Note that macOS actually does decomposing unicode normalisation on file names, I guess because it makes handling case-insensitivity easier. (Just doing ascii case insensitivity also handles o+diaresis, but not the ö codepoint)
https://developer.apple.com/library/mac/qa/qa1235/_index.html https://developer.apple.com/library/mac/qa/qa1235/_index.htm...