2 ms·
This library does not come even close to doing what it claims to. This is excusable because it is actually a very difficult problem with no simple solution. De
by dmitrygr 9y ago
This library does not come even close to doing what it claims to.
This is excusable because it is actually a very difficult problem with no simple solution. Depending on your hardware and software, it may have no guaranteed solution at all.
Let's start with the fact that modern disc controllers cache megabytes of data and there exists no way to force them to write it out. There exist a few ways to suggest this. Some will listen, some will lie. SSDs even worse in this respect, they cache hundreds of megabytes, and while they're doing garbage collection, even more data may be touched and modified. Depending on how good their algorithms are, you may lose the data that you are writing if they're powered off suddenly, or entirely unrelated other data, or perhaps a little of each. There is no standard way to tell an SSD to flush its data to actual Flash either, of course. In fact, depending the the logic in the FTL in use, this may not even be possible to do quickly (say, you discover a few bad blocks during garbage collection and need to relocate a lot of data suddenly). This changes even across firmware updates.
And then we arrive at modern file systems. The default mount options generally only sync metadata to disc no matter how many times you call the sync system call. All the dirty pages containing actual file data will be written whenever the system damn well feels like it. And of course, as mentioned above, they're only written to the volatile RAM inside the disk controller.
This is actually an insanely complicated problem. If you want to see how it is solved professionally, take a look at a databases. People who work on sqlite spent countless hundreds of hours making stuff like this work reliably, or break in ways that are predictable and recoverable. Generally this involves extreme amounts of journaling, sometimes multiple levels of journals. Oftentimes there is a whole lot of os-specific quirks handling to fix the differences between what the OSs promise functions do and the reality.
This is actually why, if you ever start needing to "write things reliably", or "store more than a few of something in a file, and update it", and you can spare the code size, just use sqlite for your storage. Because there's a whole lot of problems you're going to hit that they have already solved for you.