7 ms·

The rename is atomic, the data being in the file is the problem, there is no garuntee unless you fsync.

trav4225 10y ago
Sorry, would you mind clarifying? "The data being in the file"? The way I understand the article's proposal is this: 1. Create new symlink pointing to desired file (assumed to already exist in a stable state). 2. Move new symlink over old symlink.
- asdfaoeu 10y ago
  Symlinks are just special files with a the contents being the link contents. His argument is that it is not atomic when considering a server crash. But I don't think that matters anyway.
  - geocar 10y ago
    Symlinks are not files, they do not have inodes and you cannot open them to fsync their contents. They exist as directory entries with a small in-directory content only, thus syncing the directory they exist in is sufficient to persist them.
    - icebraining 10y ago
      Symlinks do have inodes, and their content (the link information) will be stored in that inode structure, if it fits, not in the directory. See e.g. https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#Symbolic_Links https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#Symb...
      - geocar 10y ago
        Sorry, you're right, however this is an implementation detail, and nothing to do with POSIX which did not define any use for the `d_ino` field for a symbolic link[1]. I think this is made unnecessarily difficult by Linux/ext4 calling something an "inode" that is not what UNIX traditionally called an inode (or what POSIX calls a "file serial number"). I think it is more useful to think of the symbolic link as "inside" the directory it is in, because this tells the programmer what to do (and what to sync: the directory). [1]: http://pubs.opengroup.org/onlinepubs/009695399/functions/readdir_r.html http://pubs.opengroup.org/onlinepubs/009695399/functions/rea...
        yuubi 10y ago
        An inode, traditionally, is the per-file data structure on disk (or in memory) that contains the user/group IDs, file type (plain/directory/device/etc; not jpeg/png/executable/etc), permissions, file size, and the location of the content (either a list of block numbers, or the address of a block containing the list if it wouldn't all fit into the fixed-size inode), among other things. These structures were identified by a number. This is an implementation detail not specified by POSIX. A directory, historically (on systems with 14- or 30-character file name limits like we had in the 1990s, at least 4.2 or 4.3 BSD-derived), consisted of a file containing a list of (16-bit inode number, file name) structs, thus making a nice round 16 or 32 bytes per directory entry, evenly dividing a 512-byte block. On some older systems you could see this structure by opening and reading a directory with open and read (I think every system everywhere prohibited writes to a directory because modifying a directory would allow you to access files in unreadable directories, uncleanly delete any file given its inode number, etc). The only special thing about a directory on disk was that its inode said it was a dir, so the filesystem code would trust it as a source of inode numbers and otherwise use it as a dir. Some of the 14-byte-name systems supported symlinks. As there was no space in the directory entry for anything other than an inode number, the link contents would have to be accessed through an inode. (I think I've heard of systems that could store very small file contents like typical symlink target names in the inode instead of allocating a data block, but can't name one). The number you see in ls -i, which is sometimes called just the "inode" (and apparently also called the "file serial number"), is the number in the directory entry that's used to find the inode. I guess someone somewhere had or intended to have a system that could store symlinks in the directory instead of consuming an inode.
    - josefbacik 10y ago
      You are confusing hardlinks with symlinks. They are similar but not the same. Symlinks in Linux most certainly get their own inode and can contain data blocks.
    - caf 10y ago
      The owning uid/gid has always been stored in the inode rather than the directory entry. Did symlinks traditionally have identical ownership to the containing directory?
      - geocar 10y ago
        Yes. IIRC this occurred sometime around SUSv2 (maybe a little earlier) when many filesystem implementations were storing the contents of the symbolic link physically outside of the directory listing.
Rapzid 10y ago
Per the documentation this seems to be the case. However, I'd be curious to know if the implementation of the symlink syscall does or does not fsync the new file while in kernel space... The fact that it does or does not should actually be in the documentation IMHO; probably a good enhancement request.
- geocar 10y ago
  Symlinks are directory entries, not files, so you need to sync the directory that contains the link. POSIX does (for some strange reason) permit a symbolic link to have an "inode" (well, d_ino), but there is no way to open this "inode", and no UNIX implementations to my knowledge do this.
  - Rapzid 10y ago
    I'm not sure if this is picking nits, but symlinks have their own inode number.. This ought to classify them as a file. HOWEVER, looking into it it seems that the target may actually be stored in the inode. This mean the symlink, though a file, has no contents and thus requires no fsync. Does this sound about right?
    - geocar 10y ago
      > I'm not sure if this is picking nits, but symlinks have their own inode number.. No they don't. First of all, POSIX doesn't use the term "inode number" but "file serial number" which is silly wankery since there's no way to access a file using its "file serial number". They might as well say it was "implementation defined". Secondly, POSIX used to specify[1] that their inode was "unspecified" because UNIX systems don't store symbolic links in separate inodes, but in the directory that contains them. Now, POSIX specifies symbolic links do have a "file serial number"[2], which could be implemented as a separate file block (confusingly Linux calls this an "inode" which has nothing to do with what UNIX called an inode -- a better term might be "virtual inode" but Linux uses that for something else entirely...) To this end, I think the only sensible interpretation is the original one: UNIX symbolic links don't have inodes, and POSIX symbolic links might as well have a "hash" of the file contents in the "file serial number" since you can't do anything with it anyway. [1]: http://pubs.opengroup.org/onlinepubs/009695399/functions/readdir_r.html http://pubs.opengroup.org/onlinepubs/009695399/functions/rea... [2]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/readdir_r.html http://pubs.opengroup.org/onlinepubs/9699919799/functions/re... > HOWEVER, looking into it it seems that the target may actually be stored in the inode. This mean the symlink, though a file, has no contents and thus requires no fsync. Does this sound about right? You have to fsync() the directory that contains the symlink.

Sorry, would you mind clarifying? "The data being in the file"? The way I understand the article's proposal is this: 1. Create new symlink pointing to desired file (assumed to already exist in a stable state). 2. Move new symlink over old symlink.

asdfaoeu 10y ago
Symlinks are just special files with a the contents being the link contents. His argument is that it is not atomic when considering a server crash. But I don't think that matters anyway.
- geocar 10y ago
  Symlinks are not files, they do not have inodes and you cannot open them to fsync their contents. They exist as directory entries with a small in-directory content only, thus syncing the directory they exist in is sufficient to persist them.
  - icebraining 10y ago
    Symlinks do have inodes, and their content (the link information) will be stored in that inode structure, if it fits, not in the directory. See e.g. https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#Symbolic_Links https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#Symb...
    - geocar 10y ago
      Sorry, you're right, however this is an implementation detail, and nothing to do with POSIX which did not define any use for the `d_ino` field for a symbolic link[1]. I think this is made unnecessarily difficult by Linux/ext4 calling something an "inode" that is not what UNIX traditionally called an inode (or what POSIX calls a "file serial number"). I think it is more useful to think of the symbolic link as "inside" the directory it is in, because this tells the programmer what to do (and what to sync: the directory). [1]: http://pubs.opengroup.org/onlinepubs/009695399/functions/readdir_r.html http://pubs.opengroup.org/onlinepubs/009695399/functions/rea...
      - yuubi 10y ago
        An inode, traditionally, is the per-file data structure on disk (or in memory) that contains the user/group IDs, file type (plain/directory/device/etc; not jpeg/png/executable/etc), permissions, file size, and the location of the content (either a list of block numbers, or the address of a block containing the list if it wouldn't all fit into the fixed-size inode), among other things. These structures were identified by a number. This is an implementation detail not specified by POSIX. A directory, historically (on systems with 14- or 30-character file name limits like we had in the 1990s, at least 4.2 or 4.3 BSD-derived), consisted of a file containing a list of (16-bit inode number, file name) structs, thus making a nice round 16 or 32 bytes per directory entry, evenly dividing a 512-byte block. On some older systems you could see this structure by opening and reading a directory with open and read (I think every system everywhere prohibited writes to a directory because modifying a directory would allow you to access files in unreadable directories, uncleanly delete any file given its inode number, etc). The only special thing about a directory on disk was that its inode said it was a dir, so the filesystem code would trust it as a source of inode numbers and otherwise use it as a dir. Some of the 14-byte-name systems supported symlinks. As there was no space in the directory entry for anything other than an inode number, the link contents would have to be accessed through an inode. (I think I've heard of systems that could store very small file contents like typical symlink target names in the inode instead of allocating a data block, but can't name one). The number you see in ls -i, which is sometimes called just the "inode" (and apparently also called the "file serial number"), is the number in the directory entry that's used to find the inode. I guess someone somewhere had or intended to have a system that could store symlinks in the directory instead of consuming an inode.
  - josefbacik 10y ago
    You are confusing hardlinks with symlinks. They are similar but not the same. Symlinks in Linux most certainly get their own inode and can contain data blocks.
  - caf 10y ago
    The owning uid/gid has always been stored in the inode rather than the directory entry. Did symlinks traditionally have identical ownership to the containing directory?
    - geocar 10y ago
      Yes. IIRC this occurred sometime around SUSv2 (maybe a little earlier) when many filesystem implementations were storing the contents of the symbolic link physically outside of the directory listing.

Rapzid 10y ago

Per the documentation this seems to be the case. However, I'd be curious to know if the implementation of the symlink syscall does or does not fsync the new file while in kernel space... The fact that it does or does not should actually be in the documentation IMHO; probably a good enhancement request.

geocar 10y ago
Symlinks are directory entries, not files, so you need to sync the directory that contains the link. POSIX does (for some strange reason) permit a symbolic link to have an "inode" (well, d_ino), but there is no way to open this "inode", and no UNIX implementations to my knowledge do this.
- Rapzid 10y ago
  I'm not sure if this is picking nits, but symlinks have their own inode number.. This ought to classify them as a file. HOWEVER, looking into it it seems that the target may actually be stored in the inode. This mean the symlink, though a file, has no contents and thus requires no fsync. Does this sound about right?
  - geocar 10y ago
    > I'm not sure if this is picking nits, but symlinks have their own inode number.. No they don't. First of all, POSIX doesn't use the term "inode number" but "file serial number" which is silly wankery since there's no way to access a file using its "file serial number". They might as well say it was "implementation defined". Secondly, POSIX used to specify[1] that their inode was "unspecified" because UNIX systems don't store symbolic links in separate inodes, but in the directory that contains them. Now, POSIX specifies symbolic links do have a "file serial number"[2], which could be implemented as a separate file block (confusingly Linux calls this an "inode" which has nothing to do with what UNIX called an inode -- a better term might be "virtual inode" but Linux uses that for something else entirely...) To this end, I think the only sensible interpretation is the original one: UNIX symbolic links don't have inodes, and POSIX symbolic links might as well have a "hash" of the file contents in the "file serial number" since you can't do anything with it anyway. [1]: http://pubs.opengroup.org/onlinepubs/009695399/functions/readdir_r.html http://pubs.opengroup.org/onlinepubs/009695399/functions/rea... [2]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/readdir_r.html http://pubs.opengroup.org/onlinepubs/9699919799/functions/re... > HOWEVER, looking into it it seems that the target may actually be stored in the inode. This mean the symlink, though a file, has no contents and thus requires no fsync. Does this sound about right? You have to fsync() the directory that contains the symlink.