5 ms·
1. A cosmic ray storm turned all ASCII charactees into ECBDIC 2. Lightning struck the 12 V feed and upped the voltage to 10 MV, turning all 0 and 1’s into 6’s
by oxymoron 6y ago
1. A cosmic ray storm turned all ASCII charactees into ECBDIC
2. Lightning struck the 12 V feed and upped the voltage to 10 MV, turning all 0 and 1’s into 6’s
3. Someone spilled a New England Pale Ale on the server
4. The process was assinated by the mysterious killer only known from his modus operandi of leaving OOM written in blood across the syslog
5. Birds nested within the server and fed all the SATA cables to their babies
Seriously though, disk writes aren’t atomic.
- lalaland1125 6y agoFile renames are atomic. This is a solved problem: 1. Write your updates to a copy of the file. 2. Do an atomic rename of that copy to the original.
- Zarel 6y agoRenames aren't atomic on crash. https://danluu.com/file-consistency/ https://danluu.com/file-consistency/
- jclulow 6y agoThey are if you perform the correct sequence of fsync operations on both the file and the directories, and use a file system which is correctly implemented.
- chrismorgan 6y agoRenames aren’t atomic on crash.
- badsectoracula 6y agoWhat crash? The article linked above never explains that part, it only assumes that it will happen. From the code it sounds as if the crash can happen in the OS itself (but then the entire kernel will crash). At that point things are completely outside your control and you might as well running on broken hardware.
- mpweiher 6y agoHmm...unlike the rest of the post, he just asserts this without support. His sources seem to disagree, certainly with that kind of blanket statement. For example: Our study takes a pessimistic view of file-system behavior; for example, we even consider the case where renames are not atomic on a system crash. So this is clearly considered an outlier/unusual. (e.g., a single 512-byte write or file rename operation are guaranteed to be atomic by many current file systems when running on a hard-disk drive) [https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-pillai.pdf https://www.usenix.org/system/files/conference/osdi14/osdi14...] I remember reading quite a bit about the (performance reducing) lengths filesystems go to in order to ensure consistency of directory entries even in case of a crash, and for example how "soft updates" were introduced to accomplish the same consistency with less of a performance degradation. Looking at it from another angle, if you are running on top of a filesystem that cannot keep itself consistent, then you are SOL, there really isn't anything you can do to mitigate. Just like we can't guarantee that we will be able to persist data that's in memory to disk if the OS is free to kill us at any time. "Best effort" it is, which means getting the data to disk as quickly as possible and not corrupting what is there.
- fanf2 6y ago1. Write your updates to a copy of the file. 1a. fsync() the file to ensure the contents are durable. 2. Do an atomic rename of that copy to the original. 2a. fsync() the directory to ensure the rename is durable.
- fiddlerwoaroof 6y agoOr just use a database like SQLite or Postgres that has developers dedicated to solving this problem.
- badsectoracula 6y agoDisk writes aren't atomic but you know if they failed or not (unless your OS is lying to you but that is a problem with the OS, not your code).