3 ms·
I've somewhat researched the topic of "durability + harddisks/SSDs" for a while since I was interested in writing a simple Key-Value store. The simple answer i
by zaarn 9y ago
I've somewhat researched the topic of "durability + harddisks/SSDs" for a while since I was interested in writing a simple Key-Value store.
The simple answer is that "fsync" is basically the best option you have. The hardware controller tells the OS "sure, I've written all this" and the OS has no real mechanism to verify this.
Worse even, some cheaper SSDs an HDDs do reorder writes around fsync's, in one case a bad HDD was able to corrupt ZFS(!) (reproducible) when loosing power during a write.
The fun thing is also that filesystems themselves aren't much better either. Ext4&co. sometimes don't propagate write errors at all, so in the remote event that the HDD controllers lowers itself down the plebian OS to tell it that it may have lost a byte or gigabyte, the filesystem is probably going to ignore it since the write was already buffered and fsync'd.
The only real reason why computers aren't spontaneously combusting is that most HDDs and SSDs don't do this, even in consumer space and an unexpected loss of power is rare to also meet some critical write. (ext4's physical journaling also helps a lot to make it resistant) If you want a full taste of how it works out, get a Raspberry Pi, load in a super cheap microSD and unplug the power a couple times while running a browser or other disk-heavy load.
- jeffreyrogers 9y agoInteresting. I'm surprised that even cheap SSDs and HDDS will reorder the writes. I've heard that most enterprise and higher end hard drives don't lie about fsync (actually it's a bit more subtle, I've heard some do, but they have enough energy stored in capacitors to finish writing if power fails).
- zaarn 9y agoThey're cheap and the vendors like to pretty up the speed charts by ensuring any speed test will mostly hit the cache, even if it comes at the cost of loosing data during a power loss. Some enterprise SSD/HDD do have large enough capacitors to quickly commit the cache to a flash version or write it out, though usually you have a battery backed RAID controller for that.