3 ms·

The most commonly used filesystems on Linux, etx4 and XFS don't do checksums.

by taffer 2y ago

alwayslikethis 2y ago
And if you care about data integrity, you shouldn't be using these. It's zfs, btrfs if you don't want to deal with the licensing issue, or bcachefs (though it is pretty new at this point)
- jcul 2y ago
  First I've heard of bcachefs, I thought you were referring to bcache, which I've used a few years ago. So was a little confused. After some googling it seems bcachefs is related to bcache but is a fully fledged filesystem rather than just a cache. Looks interesting.
- sgbeal 2y ago
  > And if you care about data integrity, you shouldn't be using these [ext4 or XFS] FUD. i've been using ext4 since it came out (and ext2 and ext3 before that), on dozens of computers, and have literally never once had any corruption with it. With XFS i had one instance of filesystem corruption way back in the early 2000's. i've read of _many_ more cases of data loss with ZFS and Btrfs on Linux than i have read about due to ext4 corruption.
  - matja 2y ago
    How do you know you never had corruption?
  - OskarS 2y ago
    If you truly believe this, there’s no reason you would want to add checksumming to SQLite either. Either the data you write can be corrupted or it can’t.
- taffer 2y ago
  That's poor advice. Use a database with checksums enabled like DB2, Oracle, Postgres, SQL Server or SQLite with the Checksum VFS Shim. TFA recommends the latter for good reason. Running a database on ZFS results in reduced throughput, while Btrfs and Bcachefs are outright performance disasters. [1][2] [1] https://www.enterprisedb.com/blog/postgres-vs-file-systems-performance-comparison https://www.enterprisedb.com/blog/postgres-vs-file-systems-p... [2] https://www.phoronix.com/review/bcachefs-benchmarks-linux67 https://www.phoronix.com/review/bcachefs-benchmarks-linux67
chasil 2y ago
Newer versions of XFS are checksumming metadata. That would not cover sqlite content. https://blogs.oracle.com/linux/post/formatting-an-xfs-filesystem https://blogs.oracle.com/linux/post/formatting-an-xfs-filesy...

The most commonly used filesystems on Linux, etx4 and XFS don't do checksums.

by taffer 2y ago

The most commonly used filesystems on Linux, etx4 and XFS don't do checksums.

And if you care about data integrity, you shouldn't be using these. It's zfs, btrfs if you don't want to deal with the licensing issue, or bcachefs (though it is pretty new at this point)

jcul 2y ago
First I've heard of bcachefs, I thought you were referring to bcache, which I've used a few years ago. So was a little confused. After some googling it seems bcachefs is related to bcache but is a fully fledged filesystem rather than just a cache. Looks interesting.
sgbeal 2y ago
> And if you care about data integrity, you shouldn't be using these [ext4 or XFS] FUD. i've been using ext4 since it came out (and ext2 and ext3 before that), on dozens of computers, and have literally never once had any corruption with it. With XFS i had one instance of filesystem corruption way back in the early 2000's. i've read of _many_ more cases of data loss with ZFS and Btrfs on Linux than i have read about due to ext4 corruption.
- matja 2y ago
  How do you know you never had corruption?
- OskarS 2y ago
  If you truly believe this, there’s no reason you would want to add checksumming to SQLite either. Either the data you write can be corrupted or it can’t.
taffer 2y ago
That's poor advice. Use a database with checksums enabled like DB2, Oracle, Postgres, SQL Server or SQLite with the Checksum VFS Shim. TFA recommends the latter for good reason. Running a database on ZFS results in reduced throughput, while Btrfs and Bcachefs are outright performance disasters. [1][2] [1] https://www.enterprisedb.com/blog/postgres-vs-file-systems-performance-comparison https://www.enterprisedb.com/blog/postgres-vs-file-systems-p... [2] https://www.phoronix.com/review/bcachefs-benchmarks-linux67 https://www.phoronix.com/review/bcachefs-benchmarks-linux67

chasil 2y ago

Newer versions of XFS are checksumming metadata. That would not cover sqlite content. https://blogs.oracle.com/linux/post/formatting-an-xfs-filesystem https://blogs.oracle.com/linux/post/formatting-an-xfs-filesy...