3 ms·

Then what happens when you mount it with the utf-8 option on one machine, and without it on another. The one without the utf-8 options writes a filename that br

Then what happens when you mount it with the utf-8 option on one machine, and without it on another. The one without the utf-8 options writes a filename that breaks utf-8 encoding. You then try to mount it again with the utf-8 option. What happens then?

gpderetta 3y ago
Not only mount. What happens when you unpack an archive that uses non-fully utf-8 compliant names. Or checkout the history of a git repo with non-utf-8 filenames. Network filesystems are also a source of pain. And even if you settle for utf-8 you'll still have to deal with differences with filesystems automatically canonicalizing names or being case insensitive.
- patrec 3y ago
  > What happens when you unpack an archive that uses non-fully utf-8 compliant names. Exactly what you would want to happen (this is the actual error message you will get, BTW)? > (cd /mnt/funkyfilesystem && touch $'\370' && tar cf my-bad-file.tar $'\370') > tar xf /mnt/funkyfilesystem/my-bad-file.tar tar: \370: Cannot open: Invalid or incomplete multibyte or wide character tar: Exiting with failure status due to previous errors Or are you trying to tell me that your life isn't complete without tar or a git checkout silently dropping some nonsense byte sequences into your filesystem? In both cases you can map the names to something saner explicitly and continue with whatever you were doing, or checkout to e.g. a tmpfs where you flipped off utf8-enforcement. BTW, unless you work with fairly unusual git repos or tar archives on a regular basis, this is very unlikely to ever happen to you. > And even if you settle for utf-8 you'll still have to deal with differences with filesystems automatically canonicalizing names or being case insensitive. (Case-preserving you mean? I think truly case-insensitive died with 8.3 DOS filenames). These are also annoying, but a much more minor issue, e.g. you don't need to create a new weird string type just because of them.
  - gpderetta 3y ago
    Your tar example appear to Work On My Machine. Probably I don't have a funky enough filesystem. Of course if the filesystem itself doesn't support invalid unicode names, there isn't much the os or tools can do. But hey, if enough tools and applications stop supporting non-utf-8 names, it is possible that in a a few decades you might get what you want as de-facto non-utf8 files will just disappear.
patrec 3y ago
I agree it's something that needs to be dealt with, but can you name a plausible scenario in which this would actually happen and in which the desired behavior would be that you get silently some garbage filename written to your disk? Let's say you get an EILSEQ on accessing the dirent with readdir and now have to debug how you got some corrupted filename you clearly didn't want in the first place, remount and fix it. How is that worse than having the corrupted filename and not knowing about it before something more insidious happens?

gpderetta 3y ago

Not only mount. What happens when you unpack an archive that uses non-fully utf-8 compliant names. Or checkout the history of a git repo with non-utf-8 filenames. Network filesystems are also a source of pain. And even if you settle for utf-8 you'll still have to deal with differences with filesystems automatically canonicalizing names or being case insensitive.

patrec 3y ago
> What happens when you unpack an archive that uses non-fully utf-8 compliant names. Exactly what you would want to happen (this is the actual error message you will get, BTW)? > (cd /mnt/funkyfilesystem && touch $'\370' && tar cf my-bad-file.tar $'\370') > tar xf /mnt/funkyfilesystem/my-bad-file.tar tar: \370: Cannot open: Invalid or incomplete multibyte or wide character tar: Exiting with failure status due to previous errors Or are you trying to tell me that your life isn't complete without tar or a git checkout silently dropping some nonsense byte sequences into your filesystem? In both cases you can map the names to something saner explicitly and continue with whatever you were doing, or checkout to e.g. a tmpfs where you flipped off utf8-enforcement. BTW, unless you work with fairly unusual git repos or tar archives on a regular basis, this is very unlikely to ever happen to you. > And even if you settle for utf-8 you'll still have to deal with differences with filesystems automatically canonicalizing names or being case insensitive. (Case-preserving you mean? I think truly case-insensitive died with 8.3 DOS filenames). These are also annoying, but a much more minor issue, e.g. you don't need to create a new weird string type just because of them.
- gpderetta 3y ago
  Your tar example appear to Work On My Machine. Probably I don't have a funky enough filesystem. Of course if the filesystem itself doesn't support invalid unicode names, there isn't much the os or tools can do. But hey, if enough tools and applications stop supporting non-utf-8 names, it is possible that in a a few decades you might get what you want as de-facto non-utf8 files will just disappear.

patrec 3y ago

I agree it's something that needs to be dealt with, but can you name a plausible scenario in which this would actually happen and in which the desired behavior would be that you get silently some garbage filename written to your disk? Let's say you get an EILSEQ on accessing the dirent with readdir and now have to debug how you got some corrupted filename you clearly didn't want in the first place, remount and fix it. How is that worse than having the corrupted filename and not knowing about it before something more insidious happens?