4 ms·

Oh and one warning regarding UTF-8 everywhere, use it but perhaps dont force it on a filesystem level. I have a file in a zfs filesystem with utf-8 and normD wh

by midjji 5y ago

Oh and one warning regarding UTF-8 everywhere, use it but perhaps dont force it on a filesystem level. I have a file in a zfs filesystem with utf-8 and normD which everytime I try to delete it remains, and if I copy/move it in the filesystem it duplicates in size and if I try to copy it to someplace else, that fails. It has some wierd seemingly cryllic filename too long to be shown and changing with zfs version numbers and possibly randomly. I think it was a meme image originally, but its taking up 135GB of tank space by now. I mostly keep it for fun, hoping zfs will eventually either declare it broken or fix it so I can see what it was. I'd share it but the tank is huge, and I never figured out a way to move or read it without dd ing the entire tank.

nly 5y ago
Can you explain this? Aren't file names on unix platforms essentially blobs, sans a few delimiters ('/', '\0', maybe '\')?
- Filligree 5y ago
  ZFS has a 'utf8only' property which, in principle, lets you constrain all filenames on a dataset to be utf-8... only. It does not otherwise change the API, but it should make the creation of a non-utf8 file into an I/O error. This defaults to off, so by default you're right. Apparently the GP found a bug in this code. I'd be interested in seeing the github issue.
  - midjji 5y ago
    With utf8only you have to specify a way to convert non utf8 to utf8, I think the problem is a bug in the normD converter.
- AnIdiotOnTheNet 5y ago
  IIRC the only character disallowed in EXT2 at least is '/'. Interesting things happen if you manage to get '/' into a file name, which can be done with a hex editor or a broken Windows EXT2 file browser in my case.
  - midjji 5y ago
    Oh that must have hurt. Heh I remember back when linux supported longer names than windows and people would come in with huge important and utterly unversioned projects which someone copied/mved on windows with some buggy forcy linux script.
- midjji 5y ago
  I really cant. I have no idea how it works or why.
  - jstimpfle 5y ago
    Do you happen to use a GUI file manager? I've seen one of those handle "incorrectly encoded" filenames very badly. I'd assume that most filesystems don't care about encoding (and I think they really shouldn't), but there are exceptions in non-Linux worlds as well.
    - midjji 5y ago
      No its on a x free server. rm, mv, cp dont work. I agree the low level stuff should never care, they are always just binblobs in kernel, and should be. Though there are certainly still bugs caused by encoding changes of some conf file gets read by it. The advantage of forcing it on a filesystemlevel was that the gigantic pile of old material with varying encoding at least was consistently off between locales rather than working for the people who used it most, and fail in the rare but critical cases for the people who rarely worked on it. Basically forcing people to fix their shit.

by midjji 5y ago

nly 5y ago

Can you explain this? Aren't file names on unix platforms essentially blobs, sans a few delimiters ('/', '\0', maybe '\')?

Filligree 5y ago
ZFS has a 'utf8only' property which, in principle, lets you constrain all filenames on a dataset to be utf-8... only. It does not otherwise change the API, but it should make the creation of a non-utf8 file into an I/O error. This defaults to off, so by default you're right. Apparently the GP found a bug in this code. I'd be interested in seeing the github issue.
- midjji 5y ago
  With utf8only you have to specify a way to convert non utf8 to utf8, I think the problem is a bug in the normD converter.
AnIdiotOnTheNet 5y ago
IIRC the only character disallowed in EXT2 at least is '/'. Interesting things happen if you manage to get '/' into a file name, which can be done with a hex editor or a broken Windows EXT2 file browser in my case.
- midjji 5y ago
  Oh that must have hurt. Heh I remember back when linux supported longer names than windows and people would come in with huge important and utterly unversioned projects which someone copied/mved on windows with some buggy forcy linux script.
midjji 5y ago
I really cant. I have no idea how it works or why.
- jstimpfle 5y ago
  Do you happen to use a GUI file manager? I've seen one of those handle "incorrectly encoded" filenames very badly. I'd assume that most filesystems don't care about encoding (and I think they really shouldn't), but there are exceptions in non-Linux worlds as well.
  - midjji 5y ago
    No its on a x free server. rm, mv, cp dont work. I agree the low level stuff should never care, they are always just binblobs in kernel, and should be. Though there are certainly still bugs caused by encoding changes of some conf file gets read by it. The advantage of forcing it on a filesystemlevel was that the gigantic pile of old material with varying encoding at least was consistently off between locales rather than working for the people who used it most, and fail in the rare but critical cases for the people who rarely worked on it. Basically forcing people to fix their shit.