3 ms·
Broken expectations are a cause. For instance, when ext4 gained traction, some users found a number of zeroed-out files after a crash. Why? Because the progra
by espadrine 8y ago
Broken expectations are a cause.
For instance, when ext4 gained traction, some users found a number of zeroed-out files after a crash.
Why? Because the programs managing those files updated them by truncating them and then writing them.
Why? Because fopen() has no "replace file" mode, even though the "w" mode seems to do just that as long as the system doesn't crash (it does O_WRONLY|O_CREAT|O_TRUNC).
Why? Because file systems used to not have delayed allocation when fopen() was created. ext4 introduced the concept to the ext family.
Why? Because ext4 was designed to improve performance compared to ext3, and that change does that without breaking POSIX.
Why? Because, even with ext3, the only way to correctly implement "replace file" on POSIX is to write the replacement file on the same directory, fsync() the file, close the file descriptor, fsync() the directory, and rename the file.
At that point, the whys have shifted the blame through userland programmers, POSIX fopen(), ext4, and back to userland programmers. Depending on where you stop, the solution can be:
1. fix all userland programs,
2. change ext4 to force sync upon closing a truncated file's descriptor (which got implemented),
3. upgrade the POSIX standard to have a dedicated "replace file" function call.
The truth is, userland is buggy, and the POSIX standard is arcane, so they both share blame. (But ext4, which was correctly implemented, is the only thing that got patched.)
http://dream.thunk.org/tytso/blog/2009/03/12/delayed-allocation-and-the-zero-length-file-problem/ http://dream.thunk.org/tytso/blog/2009/03/12/delayed-allocat...
- grok2 8y agoTo me it seems like ext4 was the problem here -- it tried to provide a feature that wasn't adequately supported by the underlying OS.
- espadrine 8y agoDelayed allocation was adequately supported. Implementing it in ext4 increased the risk of zeroing out a file from “having all files updated in the past 20ms be zeroed out” to “having all files updated in the past 4 seconds be zeroed out” in the event of a system crash.