7 ms·

Technically, there is nothing besides "all data written since the previous fsync" since the previous fsync already wrote all the outstanding data at that point

Technically, there is nothing besides "all data written since the previous fsync" since the previous fsync already wrote all the outstanding data at that point in time. (Even if it actually failed.) I.e., there is never any to-be-written data remaining after fsync returns. Everything that was queued to be flushed was flushed and dequeued. Whether any particular write failed or not doesn't change that fact.

mehrdadn 8y ago
> the previous fsync already wrote all the outstanding data at that point in time. (Even if it actually failed.) I'm sorry, but this... just doesn't make sense. An event can't both happen and also fail to happen. Also, the queue business is an implementation detail. Our debate is over what the spec mandates, not how a particular implementation behaves.
- hyc_symas 8y ago
  The fsync() function shall request that all data for the open file descriptor named by fildes is to be transferred to the storage device associated with the file described by fildes. The nature of the transfer is implementation-defined. The fsync() function shall not return until the system has completed that action or until an error is detected. All of the data was transferred to the device. The device may have failed to persist some or all of what was transferred, but it all got transferred. There's no mention of the OS retrying, or leaving the system in a state that a subsequent fsync can retry from where it left off. So you can't assume anything along those lines.
  - mehrdadn 8y ago
    So you're suggesting "storage device" here isn't the same thing as "storage medium"? i.e. that merely transferring the data to a volatile buffer on the device is also considered being transferred to the "storage device", rather than to the (nonvolatile) storage medium? By that logic even after a successful call you can't rely on the data being persisted, which goes squarely against the entire point of the function and its documentation: "The fsync() function is intended to force a physical write of data from the buffer cache, and to assure that after a system crash or other failure that all data up to the time of the fsync() call is recorded on the disk." So this interpretation is wrong...
    - hyc_symas 8y ago
      Nonsense. If the device returns "success" then everything was persisted. If it returns an error, then some/all of it was not persisted. There is no way for you to determine whether it is some or all or which. The only safe action for a user is to assume all of it failed. And by the way, there are devices out there that lie, and claim the data is successfully persisted even though it only made it into a volatile cache.
      - mehrdadn 8y ago
        Everything you said in this comment is consistent with what I've been saying. Nowhere did I suggest the user can assume data was written if an fsync call fails. I'm saying if an fsync call fails, the documentation of fsync (which I quoted multiple times) implies the next call will attempt to write data that wasn't successful it written on the last call.
        hyc_symas 8y ago
        Yeah, and we still disagree there. Nothing in the doc implies that just calling fsync() again will do anything useful, there is no implication of retryability.
        mehrdadn 8y ago
        > Nothing in the doc implies that just calling fsync() again will do anything useful "Nothing" in the doc? More like everything in the doc? It literally says "all data for the open file descriptor is to be transferred" and "If the fsync() function fails, outstanding I/O operations are not guaranteed to have been completed." Together these are literally saying that all data that wasn't transferred due to a failure in one call will be transferred in the next successful call. I could maybe buy it if the doc described a notion of "dequeueing" that was separate from writing, but it doesn't. It just talks about queued operations completing. So either they complete successfully (and are subsequently dequeued because that is common sense) or they don't. Like if your boss had assigned you tasks A, B, and then C, and then he ordered you to finish all your tasks, and you failed to do B, and then he made the same request again, you wouldn't then suddenly turn to him and say "I have nothing to do because I already popped B and C off my to-do list". You'd get fired for that (at least if you persisted) because it literally wouldn't make sense.
- geocar 8y ago
  > I'm sorry, but this... just doesn't make sense. An event can't both happen and also fail to happen. Consider a multithreaded application where a and b point to the same file. Note this isn't exactly what Postgres does: T1 T2 -------------------- -------------------- write(a,x) start write(a,x) success fsync(a) start read(b,x) start read(b,x) success fsync(a) failed write(b,y) start write(b,y) success fsync(b) start fsync(b) success y has a data dependancy on x (for example: table metadata), and the fsync() for y succeeded, so is T2 given to expect that x was recorded on disk correctly?
  - mehrdadn 8y ago
    No? It says "all data for the open file descriptor", not "all data for all open file descriptors".
    - geocar 8y ago
      The file descriptors point to the same file and can even be the same descriptor. Maybe this is clearer/simpler, writing x and y to the same offset: T1 T2 -------------------- -------------------- write(b,y) start write(b,y) success write(a,x) start write(a,x) success fsync(a) start fsync(a) failed fsync(b) start fsync(b) success At this point, T2 is probably surprised to discover the original contents of the file on the disk. Even after a successful fsync()!
      - mehrdadn 8y ago
        I'm confused, where is the reading happening that is causing T2 to be surprised? But in any case, I'm saying that if a and b are the same descriptor, then fsync(b)'s success should imply both a and b are written to disk, by the specification of fsync. I don't see where you're seeing a contradiction.

mehrdadn 8y ago

> the previous fsync already wrote all the outstanding data at that point in time. (Even if it actually failed.) I'm sorry, but this... just doesn't make sense. An event can't both happen and also fail to happen. Also, the queue business is an implementation detail. Our debate is over what the spec mandates, not how a particular implementation behaves.

hyc_symas 8y ago
The fsync() function shall request that all data for the open file descriptor named by fildes is to be transferred to the storage device associated with the file described by fildes. The nature of the transfer is implementation-defined. The fsync() function shall not return until the system has completed that action or until an error is detected. All of the data was transferred to the device. The device may have failed to persist some or all of what was transferred, but it all got transferred. There's no mention of the OS retrying, or leaving the system in a state that a subsequent fsync can retry from where it left off. So you can't assume anything along those lines.
- mehrdadn 8y ago
  So you're suggesting "storage device" here isn't the same thing as "storage medium"? i.e. that merely transferring the data to a volatile buffer on the device is also considered being transferred to the "storage device", rather than to the (nonvolatile) storage medium? By that logic even after a successful call you can't rely on the data being persisted, which goes squarely against the entire point of the function and its documentation: "The fsync() function is intended to force a physical write of data from the buffer cache, and to assure that after a system crash or other failure that all data up to the time of the fsync() call is recorded on the disk." So this interpretation is wrong...
  - hyc_symas 8y ago
    Nonsense. If the device returns "success" then everything was persisted. If it returns an error, then some/all of it was not persisted. There is no way for you to determine whether it is some or all or which. The only safe action for a user is to assume all of it failed. And by the way, there are devices out there that lie, and claim the data is successfully persisted even though it only made it into a volatile cache.
    - mehrdadn 8y ago
      Everything you said in this comment is consistent with what I've been saying. Nowhere did I suggest the user can assume data was written if an fsync call fails. I'm saying if an fsync call fails, the documentation of fsync (which I quoted multiple times) implies the next call will attempt to write data that wasn't successful it written on the last call.
      - hyc_symas 8y ago
        Yeah, and we still disagree there. Nothing in the doc implies that just calling fsync() again will do anything useful, there is no implication of retryability.
        mehrdadn 8y ago
        > Nothing in the doc implies that just calling fsync() again will do anything useful "Nothing" in the doc? More like everything in the doc? It literally says "all data for the open file descriptor is to be transferred" and "If the fsync() function fails, outstanding I/O operations are not guaranteed to have been completed." Together these are literally saying that all data that wasn't transferred due to a failure in one call will be transferred in the next successful call. I could maybe buy it if the doc described a notion of "dequeueing" that was separate from writing, but it doesn't. It just talks about queued operations completing. So either they complete successfully (and are subsequently dequeued because that is common sense) or they don't. Like if your boss had assigned you tasks A, B, and then C, and then he ordered you to finish all your tasks, and you failed to do B, and then he made the same request again, you wouldn't then suddenly turn to him and say "I have nothing to do because I already popped B and C off my to-do list". You'd get fired for that (at least if you persisted) because it literally wouldn't make sense.
geocar 8y ago
> I'm sorry, but this... just doesn't make sense. An event can't both happen and also fail to happen. Consider a multithreaded application where a and b point to the same file. Note this isn't exactly what Postgres does: T1 T2 -------------------- -------------------- write(a,x) start write(a,x) success fsync(a) start read(b,x) start read(b,x) success fsync(a) failed write(b,y) start write(b,y) success fsync(b) start fsync(b) success y has a data dependancy on x (for example: table metadata), and the fsync() for y succeeded, so is T2 given to expect that x was recorded on disk correctly?
- mehrdadn 8y ago
  No? It says "all data for the open file descriptor", not "all data for all open file descriptors".
  - geocar 8y ago
    The file descriptors point to the same file and can even be the same descriptor. Maybe this is clearer/simpler, writing x and y to the same offset: T1 T2 -------------------- -------------------- write(b,y) start write(b,y) success write(a,x) start write(a,x) success fsync(a) start fsync(a) failed fsync(b) start fsync(b) success At this point, T2 is probably surprised to discover the original contents of the file on the disk. Even after a successful fsync()!
    - mehrdadn 8y ago
      I'm confused, where is the reading happening that is causing T2 to be surprised? But in any case, I'm saying that if a and b are the same descriptor, then fsync(b)'s success should imply both a and b are written to disk, by the specification of fsync. I don't see where you're seeing a contradiction.