7 ms·

But there is no rigamarole. The article is just super confused. Atomicity is orthogonal to durability. rename() is atomic and always has been atomic. It is ato

But there is no rigamarole. The article is just super confused. Atomicity is orthogonal to durability. rename() is atomic and always has been atomic. It is atomic without guaranteeing durability. In fact, to guarantee durability after rename you must: 1) rename(a, b) 2) open(the parent dir) 3) fsync(the parent dir) This is to guarantee durability of the metadata written by rename() in the dirent. All of this is totally separate from guaranteeing the durability of the data in the file, which requires its own separate fsync() on the file itself. Atomicity and durability are never the same. This article is gibberish.

aidenn0 3y ago
The phrase "atomic across crashes" has a meaning that is distinct from durability. It is also different from atomic across a live system.
- throwaway09223 3y ago
  The phrase "atomic across crashes" is meaningless. Atomicity deals with the running system state at the moment a system call executes. It has nothing to do with crashes (which are not atomic in and of themselves), or what bits actually end up on disk.
  - aidenn0 3y ago
    Are you being willfully obtuse, or do you honestly believe that the word "atomicity" only applies to such a narrow context?
    - throwaway09223 3y ago
      The latter. What do you think it means?
      - rcxdude 3y ago
        Atomic simply means indivisible: you don't observe half of an atomic operation. Of course it strongly depends on where you are observing the operation from, and that does not necessarily just mean 'a running system', hence OP specifying that they wish for the operation to be atomic even when observed from a system before and after a crash.
        throwaway09223 3y ago
        Yes, but as I said above that doesn't make sense. The atomicity we're discussing here is with respect to time, and sequencing (not say size, like an atom). We say that rename() is atomic because you will either see file A or file B. There are no other possible states. With a crash, writes might not be committed. You might get an earlier filesystem state.
        aidenn0 3y ago
        > With a crash, writes might not be committed. You might get an earlier filesystem state. And the whole point of TFA is that this is perfectly fine as long as the metadata writes aren't committed before the data writes. That is: Precondition: File at path A exits with data X 1. Create file at path B 2. Write data Y to file B. 3. Close file B 4. Rename B to A 5. Crash --- As long as the contents in the file at path A are either X or Y, then we have achieve atomicity in the sense used in TFA. This is what we mean by rename acting "atomically across crashes" Note that we only care about the file at path A; all of these are fine: - File at path B exists and has data Y - File at path B exists and has garbage data - File at path B doesn't exist - File at path B exists and is of size 0
        rcxdude 3y ago
        Specifically (to hopefully clarify your point), the operation we wish to be atomic is 'replacing the mapping path A->data X with path A->data Y'. This is hoped to be done by placing data Y in path B (which does not need to be atomic )and then using the atomicity of the rename to move around the path. This then puts an ordering constraint that any observer (whether after a crash and looking at the written filesystem, or during the normal operation of the system) sees data Y going into path B before the rename B to A, and this is what POSIX does not guarantee.
dale_glass 3y ago
Okay, so why not create a single syscall to encompass all that? So that the developer can clearly say "this is what I'm trying to achieve", and accomplish it simply, and reliably? I think this would be a benefit, because first you don't need userspace developers dig into the gory details of dirent metadata durability, and second because by making this explicit it helps put pressure on filesystem developers to optimize what the users actually want.
- throwaway09223 3y ago
  Because it would accomplish nothing. What's the point? It's not more simple and there are no benefits WRT reliability. "I think this would be a benefit, because first you don't need userspace developers dig into the gory details of dirent metadata durability" This is what libraries are for, not syscalls. The problem here is you're mixing up two unrelated concepts: Durability and atomicity.

aidenn0 3y ago

The phrase "atomic across crashes" has a meaning that is distinct from durability. It is also different from atomic across a live system.

throwaway09223 3y ago
The phrase "atomic across crashes" is meaningless. Atomicity deals with the running system state at the moment a system call executes. It has nothing to do with crashes (which are not atomic in and of themselves), or what bits actually end up on disk.
- aidenn0 3y ago
  Are you being willfully obtuse, or do you honestly believe that the word "atomicity" only applies to such a narrow context?
  - throwaway09223 3y ago
    The latter. What do you think it means?
    - rcxdude 3y ago
      Atomic simply means indivisible: you don't observe half of an atomic operation. Of course it strongly depends on where you are observing the operation from, and that does not necessarily just mean 'a running system', hence OP specifying that they wish for the operation to be atomic even when observed from a system before and after a crash.
      - throwaway09223 3y ago
        Yes, but as I said above that doesn't make sense. The atomicity we're discussing here is with respect to time, and sequencing (not say size, like an atom). We say that rename() is atomic because you will either see file A or file B. There are no other possible states. With a crash, writes might not be committed. You might get an earlier filesystem state.
        aidenn0 3y ago
        > With a crash, writes might not be committed. You might get an earlier filesystem state. And the whole point of TFA is that this is perfectly fine as long as the metadata writes aren't committed before the data writes. That is: Precondition: File at path A exits with data X 1. Create file at path B 2. Write data Y to file B. 3. Close file B 4. Rename B to A 5. Crash --- As long as the contents in the file at path A are either X or Y, then we have achieve atomicity in the sense used in TFA. This is what we mean by rename acting "atomically across crashes" Note that we only care about the file at path A; all of these are fine: - File at path B exists and has data Y - File at path B exists and has garbage data - File at path B doesn't exist - File at path B exists and is of size 0
        rcxdude 3y ago
        Specifically (to hopefully clarify your point), the operation we wish to be atomic is 'replacing the mapping path A->data X with path A->data Y'. This is hoped to be done by placing data Y in path B (which does not need to be atomic )and then using the atomicity of the rename to move around the path. This then puts an ordering constraint that any observer (whether after a crash and looking at the written filesystem, or during the normal operation of the system) sees data Y going into path B before the rename B to A, and this is what POSIX does not guarantee.

dale_glass 3y ago

Okay, so why not create a single syscall to encompass all that? So that the developer can clearly say "this is what I'm trying to achieve", and accomplish it simply, and reliably? I think this would be a benefit, because first you don't need userspace developers dig into the gory details of dirent metadata durability, and second because by making this explicit it helps put pressure on filesystem developers to optimize what the users actually want.

throwaway09223 3y ago
Because it would accomplish nothing. What's the point? It's not more simple and there are no benefits WRT reliability. "I think this would be a benefit, because first you don't need userspace developers dig into the gory details of dirent metadata durability" This is what libraries are for, not syscalls. The problem here is you're mixing up two unrelated concepts: Durability and atomicity.