3 ms·
Please don't unless you've thoroughly tested in multiple cultures. Unicode is broken for Japanese; on some systems it's possible to change a setting to make it
by lmm 4y ago
Please don't unless you've thoroughly tested in multiple cultures. Unicode is broken for Japanese; on some systems it's possible to change a setting to make it broken for Chinese instead, but the only non-theoretical way to make a program that can list a directory full of Japanese filenames and a directory full of Chinese filenames correctly is to have that program be encoding-aware.
- lidavidm 4y agoI'm curious, what does 'broken' mean? Is this about the CJK merger, or something else?
- lmm 4y agoYes, CJK unification. Characters will be rendered differently from how they should be (e.g. with missing or extra strokes).
- hnfong 4y agoWhy is that a filesystem issue or a "utf8only=on" issue? If the renderer doesn't have sufficient context, it will pick one glyph to render anyway, and it could be "wrong". PS: unless your take is to say that we should just assume everything is ascii and don't ever render codepoints > 127?
- lmm 4y agoIf the filesystem can't allow parts to be in different encodings then it can't ever possibly work right. With current filesystems at least I can do "ls ~/chinese-files; LANG=ja_JP.SJIS ls ~/japanese-files". Which is not a great way of working, but it's better than a utf8only filesystem where it's physically impossible to make even that much work.
- chungy 4y agoIf you run a ZFS file system where this is a huge deal to you, then don't set utf8only. If it's set on a parent dataset, then explicitly do utf8only=off.