4 ms·

> ...but it's unique while the file exists, right? I don't think all filesystems guarantee this. Especially network filesystems.

the_mitsuhiko 1y ago
It's effectively impossible to guarantee this when you have a file system that unifies and re-exports. Network file systems being an obvious one, but overlayfs is in a similar position. Even if inodes still work nowadays they will eventually run into issues a few years down the line.
- account42 1y ago
  Then unifying file systems is not something that POSIX support and a POSIX system shouldn't do it unless it can somehow map inodes with POSIX semantics. E.g. for a network mount spanning multiple remote filesystems you could also have multiple st_dev locally.
amiga386 1y ago
That's a problem for programs that do recursive fs descent (e.g. find, tar) because they use st_dev and st_ino alone for remembering what directories they've been in. They can't just use the absolute path, because symbolic links allow for loops. find: * https://cgit.git.savannah.gnu.org/cgit/findutils.git/tree/find/sharefile.c#n51 https://cgit.git.savannah.gnu.org/cgit/findutils.git/tree/fi... * https://cgit.git.savannah.gnu.org/cgit/findutils.git/tree/find/pred.c#n900 https://cgit.git.savannah.gnu.org/cgit/findutils.git/tree/fi... tar: * https://cgit.git.savannah.gnu.org/cgit/tar.git/tree/src/create.c#n1413 https://cgit.git.savannah.gnu.org/cgit/tar.git/tree/src/crea... * https://cgit.git.savannah.gnu.org/cgit/tar.git/tree/src/names.c#n905 https://cgit.git.savannah.gnu.org/cgit/tar.git/tree/src/name... * https://cgit.git.savannah.gnu.org/cgit/tar.git/tree/src/incremen.c#n524 https://cgit.git.savannah.gnu.org/cgit/tar.git/tree/src/incr... In particular, I'm intrigued by the comment in the last link: /* With NFS, the same file can have two different devices if an NFS directory is mounted in multiple locations, which is relatively common when automounting. To avoid spurious incremental redumping of directories, consider all NFS devices as equal, relying on the i-node to establish differences. */ So GNU tar expects an inode to be unique across _all_ NFS mounts...
- the_mitsuhiko 1y ago
  You are not wrong, but the issues with tar are well known. Linus himself had this to say [1]: > Well, the fact that it hits snapshots, shows that the real problem is just "tar does stupid things that it shouldn't do". > Yes, inode numbers used to be special, and there's history behind it. But we should basically try very hard to walk away from that broken history. > An inode number just isn't a unique descriptor any more. We're not living in the 1970s, and filesystems have changed. You might still get away with it most of the time today, but it's causing more and more issues. [1]: https://lkml.iu.edu/hypermail/linux/kernel/2401.3/04127.html https://lkml.iu.edu/hypermail/linux/kernel/2401.3/04127.html
  - amiga386 1y ago
    That sounds like blaming userspace. If it's not the 1970s anymore, then update the POSIX standard with a solution that works for all OSes (including the BSDs) and can be relied upon. Definitely don't suggest a Linux-only solution for a Linux-only problem.
    - dwattttt 1y ago
      Have you checked what POSIX has to say about inode numbers? It may say less than you think.
      - amiga386 1y ago
        https://pubs.opengroup.org/onlinepubs/009696799/basedefs/sys/stat.h.html https://pubs.opengroup.org/onlinepubs/009696799/basedefs/sys... > The st_ino and st_dev fields taken together uniquely identify the file within the system. It says exactly what it ought to say.
        dwattttt 1y ago
        Issue 8 (2024, https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/sys_stat.h.html https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/sy...) relaxed the language there. st_ino and st_dev still uniquely identify a file, but it now notes that the duration it identifies it for is not indefinite. As an example, it offers that the identity of a file that's deleted could be reused.

the_mitsuhiko 1y ago

It's effectively impossible to guarantee this when you have a file system that unifies and re-exports. Network file systems being an obvious one, but overlayfs is in a similar position. Even if inodes still work nowadays they will eventually run into issues a few years down the line.

account42 1y ago
Then unifying file systems is not something that POSIX support and a POSIX system shouldn't do it unless it can somehow map inodes with POSIX semantics. E.g. for a network mount spanning multiple remote filesystems you could also have multiple st_dev locally.

amiga386 1y ago

That's a problem for programs that do recursive fs descent (e.g. find, tar) because they use st_dev and st_ino alone for remembering what directories they've been in. They can't just use the absolute path, because symbolic links allow for loops. find: * https://cgit.git.savannah.gnu.org/cgit/findutils.git/tree/find/sharefile.c#n51 https://cgit.git.savannah.gnu.org/cgit/findutils.git/tree/fi... * https://cgit.git.savannah.gnu.org/cgit/findutils.git/tree/find/pred.c#n900 https://cgit.git.savannah.gnu.org/cgit/findutils.git/tree/fi... tar: * https://cgit.git.savannah.gnu.org/cgit/tar.git/tree/src/create.c#n1413 https://cgit.git.savannah.gnu.org/cgit/tar.git/tree/src/crea... * https://cgit.git.savannah.gnu.org/cgit/tar.git/tree/src/names.c#n905 https://cgit.git.savannah.gnu.org/cgit/tar.git/tree/src/name... * https://cgit.git.savannah.gnu.org/cgit/tar.git/tree/src/incremen.c#n524 https://cgit.git.savannah.gnu.org/cgit/tar.git/tree/src/incr... In particular, I'm intrigued by the comment in the last link: /* With NFS, the same file can have two different devices if an NFS directory is mounted in multiple locations, which is relatively common when automounting. To avoid spurious incremental redumping of directories, consider all NFS devices as equal, relying on the i-node to establish differences. */ So GNU tar expects an inode to be unique across _all_ NFS mounts...

the_mitsuhiko 1y ago
You are not wrong, but the issues with tar are well known. Linus himself had this to say [1]: > Well, the fact that it hits snapshots, shows that the real problem is just "tar does stupid things that it shouldn't do". > Yes, inode numbers used to be special, and there's history behind it. But we should basically try very hard to walk away from that broken history. > An inode number just isn't a unique descriptor any more. We're not living in the 1970s, and filesystems have changed. You might still get away with it most of the time today, but it's causing more and more issues. [1]: https://lkml.iu.edu/hypermail/linux/kernel/2401.3/04127.html https://lkml.iu.edu/hypermail/linux/kernel/2401.3/04127.html
- amiga386 1y ago
  That sounds like blaming userspace. If it's not the 1970s anymore, then update the POSIX standard with a solution that works for all OSes (including the BSDs) and can be relied upon. Definitely don't suggest a Linux-only solution for a Linux-only problem.
  - dwattttt 1y ago
    Have you checked what POSIX has to say about inode numbers? It may say less than you think.
    - amiga386 1y ago
      https://pubs.opengroup.org/onlinepubs/009696799/basedefs/sys/stat.h.html https://pubs.opengroup.org/onlinepubs/009696799/basedefs/sys... > The st_ino and st_dev fields taken together uniquely identify the file within the system. It says exactly what it ought to say.
      - dwattttt 1y ago
        Issue 8 (2024, https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/sys_stat.h.html https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/sy...) relaxed the language there. st_ino and st_dev still uniquely identify a file, but it now notes that the duration it identifies it for is not indefinite. As an example, it offers that the identity of a file that's deleted could be reused.