4 ms·

Is ZFS any good? What's the current recommendation of file system? ext4, btrfs, ZFS? Is there one better suited for a laptop/desktop and a server? Is one be

by Jonnax 7y ago

Is ZFS any good? What's the current recommendation of file system? ext4, btrfs, ZFS? Is there one better suited for a laptop/desktop and a server? Is one better for RAID? Is one better for speed? Is one better for resiliency?

ssvss 7y ago
Most DBAs working with async/directIO databases recommend XFS, https://www.scylladb.com/2016/02/09/qualifying-filesystems/ https://www.scylladb.com/2016/02/09/qualifying-filesystems/
- throw0101a 7y ago
  For the topic of directIO / O_DIRECT, this is specific to databases that do their own buffering, and thus one would have double-buffering: once at the DB app layer and again at the OS VFS layer. Not all databases do buffering: Postgresql does not, and is generally built around an OS's normal I/O buffering. 2017 sub-thread on this topic: * https://news.ycombinator.com/item?id=15722708 https://news.ycombinator.com/item?id=15722708
- rhinoceraptor 7y ago
  That seems like terrible advice considering it has no silent data corruption protection. You should treat your storage hardware as if it's trying to corrupt your data.
throw0101a 7y ago
> Is ZFS any good? The presentation "ZFS: The Last Word in File Systems" from its debut in 2008 is a good introduction. Part 1 (of 3) probably best explains how different it is from just about anything else out there (even now, in 2019): * https://www.youtube.com/watch?v=NRoUC9P1PmA https://www.youtube.com/watch?v=NRoUC9P1PmA Part 2 show some extra niceties / features, and Part 3 is a demo and Q&A. Watching Part 1 at least is worth it for anyone curious; 2 and 3 are optional.
rhinoceraptor 7y ago
ZFS is 100% better for RAID and resiliency. If you care about your data, ZFS is the answer. Even if you only have a single disk with no redundancy, at the very least you'll know when your data is corrupted, so you can avoid polluting your backups with that corrupted data. It's possibly better for performance, depending on use case. If you're dealing with easily compressible data, then you can enable compression, which can speed up your reads/writes since you're reading or writing less data. You shouldn't get a performance hit on non-compressible data, since it will adjust its compression when it gets non-compressible data.
- beatgammit 7y ago
  How does it compare to BTRFS? It has many of the same features and is supported today in production (SUSE ships it today by default). I really like ZFS, and it's definitely more robust than most other filesystems, but there are multiple solutions "if you care about your data".
  - emergie 7y ago
    btrfs still hasn't got working raid567 (raidz123 in zfs) https://btrfs.wiki.kernel.org/index.php/RAID56 https://btrfs.wiki.kernel.org/index.php/RAID56 in the begining of 2019 this page had a big fat warning: The parity RAID code has '''multiple serious data-loss bugs''' in it. It should '''not''' be used for anything other than testing purposes. Someone watered it down in march, but it still doesn't look much better.

by Jonnax 7y ago

ssvss 7y ago

Most DBAs working with async/directIO databases recommend XFS, https://www.scylladb.com/2016/02/09/qualifying-filesystems/ https://www.scylladb.com/2016/02/09/qualifying-filesystems/

throw0101a 7y ago
For the topic of directIO / O_DIRECT, this is specific to databases that do their own buffering, and thus one would have double-buffering: once at the DB app layer and again at the OS VFS layer. Not all databases do buffering: Postgresql does not, and is generally built around an OS's normal I/O buffering. 2017 sub-thread on this topic: * https://news.ycombinator.com/item?id=15722708 https://news.ycombinator.com/item?id=15722708
rhinoceraptor 7y ago
That seems like terrible advice considering it has no silent data corruption protection. You should treat your storage hardware as if it's trying to corrupt your data.

throw0101a 7y ago

> Is ZFS any good? The presentation "ZFS: The Last Word in File Systems" from its debut in 2008 is a good introduction. Part 1 (of 3) probably best explains how different it is from just about anything else out there (even now, in 2019): * https://www.youtube.com/watch?v=NRoUC9P1PmA https://www.youtube.com/watch?v=NRoUC9P1PmA Part 2 show some extra niceties / features, and Part 3 is a demo and Q&A. Watching Part 1 at least is worth it for anyone curious; 2 and 3 are optional.

rhinoceraptor 7y ago

ZFS is 100% better for RAID and resiliency. If you care about your data, ZFS is the answer. Even if you only have a single disk with no redundancy, at the very least you'll know when your data is corrupted, so you can avoid polluting your backups with that corrupted data. It's possibly better for performance, depending on use case. If you're dealing with easily compressible data, then you can enable compression, which can speed up your reads/writes since you're reading or writing less data. You shouldn't get a performance hit on non-compressible data, since it will adjust its compression when it gets non-compressible data.

beatgammit 7y ago
How does it compare to BTRFS? It has many of the same features and is supported today in production (SUSE ships it today by default). I really like ZFS, and it's definitely more robust than most other filesystems, but there are multiple solutions "if you care about your data".
- emergie 7y ago
  btrfs still hasn't got working raid567 (raidz123 in zfs) https://btrfs.wiki.kernel.org/index.php/RAID56 https://btrfs.wiki.kernel.org/index.php/RAID56 in the begining of 2019 this page had a big fat warning: The parity RAID code has '''multiple serious data-loss bugs''' in it. It should '''not''' be used for anything other than testing purposes. Someone watered it down in march, but it still doesn't look much better.