3 ms·

Can anyone provide links to learn more about these misconceptions and the bugs they caused? Edit: talking about filesystem misconceptions, not fsyncgate.

sgarland 2y ago
EDIT: realized you were asking about the ext4 bugs, not the parent comment of that one. Oh well, keeping this up anyway for others. Postgres has a whole wiki page [0] about it, it's quite a read. They also link a [1] MySQL commit to fix the same issue. [0]: https://wiki.postgresql.org/wiki/Fsync_Errors https://wiki.postgresql.org/wiki/Fsync_Errors [1]: https://github.com/mysql/mysql-server/commit/8590c8e12a3374eeccb547359750a9d2a128fa6a#diff-28ed40e7ccec6683ebd46da2ca82d01c https://github.com/mysql/mysql-server/commit/8590c8e12a3374e...
praptak 2y ago
I have found a blog post from 2009 about this issue[1]. I think the "recent Ubuntu bug [that] has gotten slashdotted" is [2], so I think the most technical discussion about this would be [2]. I also found an interesting "scar tissue" from that bug in the current ext4 docs[0]: "If auto_da_alloc is enabled, ext4 will detect the replace-via-rename and replace-via-truncate patterns and force that any delayed allocation blocks are allocated such that at the next journal commit, in the default data=ordered mode, the data blocks of the new file are forced to disk before the rename() operation is committed. This provides roughly the same level of guarantees as ext3, and avoids the “zero-length” problem that can happen when a system crashes before the delayed allocation blocks are forced to disk." [0]https://docs.kernel.org/admin-guide/ext4.html https://docs.kernel.org/admin-guide/ext4.html [1] https://thunk.org/tytso/blog/2009/03/12/delayed-allocation-and-the-zero-length-file-problem/ https://thunk.org/tytso/blog/2009/03/12/delayed-allocation-a... [2] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/317781 https://bugs.launchpad.net/ubuntu/+source/linux/+bug/317781

sgarland 2y ago

EDIT: realized you were asking about the ext4 bugs, not the parent comment of that one. Oh well, keeping this up anyway for others. Postgres has a whole wiki page [0] about it, it's quite a read. They also link a [1] MySQL commit to fix the same issue. [0]: https://wiki.postgresql.org/wiki/Fsync_Errors https://wiki.postgresql.org/wiki/Fsync_Errors [1]: https://github.com/mysql/mysql-server/commit/8590c8e12a3374eeccb547359750a9d2a128fa6a#diff-28ed40e7ccec6683ebd46da2ca82d01c https://github.com/mysql/mysql-server/commit/8590c8e12a3374e...

praptak 2y ago

I have found a blog post from 2009 about this issue[1]. I think the "recent Ubuntu bug [that] has gotten slashdotted" is [2], so I think the most technical discussion about this would be [2]. I also found an interesting "scar tissue" from that bug in the current ext4 docs[0]: "If auto_da_alloc is enabled, ext4 will detect the replace-via-rename and replace-via-truncate patterns and force that any delayed allocation blocks are allocated such that at the next journal commit, in the default data=ordered mode, the data blocks of the new file are forced to disk before the rename() operation is committed. This provides roughly the same level of guarantees as ext3, and avoids the “zero-length” problem that can happen when a system crashes before the delayed allocation blocks are forced to disk." [0]https://docs.kernel.org/admin-guide/ext4.html https://docs.kernel.org/admin-guide/ext4.html [1] https://thunk.org/tytso/blog/2009/03/12/delayed-allocation-and-the-zero-length-file-problem/ https://thunk.org/tytso/blog/2009/03/12/delayed-allocation-a... [2] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/317781 https://bugs.launchpad.net/ubuntu/+source/linux/+bug/317781