6 ms·
Durability: Linux File APIs
- dfox 6y ago> Does that mean write is atomic? Technically, yes: future reads must return the entire contents of the write, or none of it. However, write is not required to be complete, and is allowed to only transfer part of the data. This is technically right interpretation of what POSIX says but totally ignores the fact that in such state the application will know that this happened and should handle that. Because in the first place the rest of the write request did not happen at all and it will not happen unless the application retries the write(2) with rest of the data. Most applications simply ignore this situation which is the reason that this situation is guaranteed to not happen on any even vaguely unix-derived OS (and exactly this is the number one reason for error messages of the "Error: Success" kind).
- the8472 6y agoand then there's mmap where you can observe changes at byte-level granularity.
- remexre 6y agoRight, I think the point is that you no longer have atomic semantics for the write as a whole.
- formerly_proven 6y agoWell until people's file descriptors happen to be pipes or sockets.
- Upvoter33 6y agobunch of academic work on this, eg. https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-zheng_mai.pdf https://www.usenix.org/system/files/conference/osdi14/osdi14... https://research.cs.wisc.edu/adsl/Publications/alice-osdi14.pdf https://research.cs.wisc.edu/adsl/Publications/alice-osdi14....
- r-r-r 6y agoSomewhat relevant is this article, https://danluu.com/file-consistency/ https://danluu.com/file-consistency/ which was discussed here a while, back.
- hnakamur 6y agoThanks for your pointer. Do you mean this thread? "Json-Base – Database built as JSON files | Hacker News" https://news.ycombinator.com/item?id=23715558 https://news.ycombinator.com/item?id=23715558
- r-r-r 6y agoI was thinking about this dedicated post, https://news.ycombinator.com/item?id=21674610 https://news.ycombinator.com/item?id=21674610 but I think I saw it multiple times here over the years. (I think there was at least one more article by danluu on this subject but this one I remembered by name, having discussed it several times with colleagues when considering using something (sqlite) less ad hoc for temporary data)
- nh2 6y agoAlso useful: https://blog.httrack.com/blog/2013/11/15/everything-you-always-wanted-to-know-about-fsync/ https://blog.httrack.com/blog/2013/11/15/everything-you-alwa...
- formerly_proven 6y agoPOSIX's atomicity guarantees relate to a running system; durability is about systems that are shut down (possibly unexpected and in an uncontrolled manner). POSIX is carefully worded to not give any durability guarantees. Naturally, writes are generally not atomic w.r.t. unexpected system maintenance and there are no ordering guarantees at all unless you give ordering hints (like fsync/fdatasync/sync_file_range) yourself, which the system (kernel, filesystem, hardware) may or may not honor.
- mehrdadn 6y ago> Durability is about systems that are shut down (possibly unexpected and in an uncontrolled manner). POSIX is carefully worded to not give any durability guarantees. What part of the POSIX specs are you reading? "The fsync() function shall request that all data for the open file descriptor named by fildes is to be transferred to the storage device associated with the file described by fildes. [...] The fsync() function is intended to force a physical write of data from the buffer cache, and to assure that after a system crash or other failure that all data up to the time of the fsync() call is recorded on the disk." https://pubs.opengroup.org/onlinepubs/9699919799/functions/fsync.html https://pubs.opengroup.org/onlinepubs/9699919799/functions/f...
- chrisseaton 6y agoHow can software possibly be expected to ensure what a physical device returns after the system restarts? I would guess that 'intended to' is there very specifically to ensure that this statement really means... nothing at all. It can fail but if it 'intended to' succeed that's enough. If your system crashes due to a power surge that breaks your hard drive... guess what it isn't going to give you back the bytes you wanted.
- mehrdadn 6y ago> How can software possibly be expected to ensure what a physical device returns after the system restarts? By giving instructions to the device that the device understands and provides guarantees for...
- tzs 6y ago> In particular, when you first create a file, you need to call fsync on the directory that contains it, otherwise it might not exist after a failure. Suppose you have created a file in a directory that is not readable. How do you get a file descriptor for the directory to call fsync on? It's an error to try to open a directory for writing.
- ekimekim 6y agoHm, from what I can tell this doesn't seem to be possible. There is one way to open a directory without read permissions, which is with O_PATH, but the resulting fd can only be used "to indicate a location in the filesystem tree and to perform operations that act purely at the file descriptor level". And indeed if you try to use this fd to fsync() you get EBADF. I guess this is just an edge case of an edge case that most people aren't concerned about. One way to work around this issue with a bare minimum of compromise to security would be a setuid program that does nothing but open its working directory then fsync it.
- nh2 6y agoPrograms that care about durability should use `openat(int dirfd, char *pathname, ...)` to get an FD on the file after after getting an FD on the parent dir anyway, instead of using `/path/to/dir/file` style paths, because otherwise there's always the chance that something in hierarchy was moved away or mounted over. In that case, you always have a handle on the directory. Opening the dir for fsyncing separately after having written the file is similar to doing close()+re-open()+fsync() on a file, which does not provide as strong guarantees as direct open()+fsync(): https://stackoverflow.com/questions/37288453/calling-fsync2-after-close2/50158433#50158433 https://stackoverflow.com/questions/37288453/calling-fsync2-...