6 ms·

What are the latest developments on this front? Still ZFS for stability, XFS for performance? Now that we have NAND storage, would we create filesystems differe

What are the latest developments on this front? Still ZFS for stability, XFS for performance? Now that we have NAND storage, would we create filesystems differently? Are there ways to make a step-change in overall performance somewhere? Disclaimer: I know nothing about filesystems..

mtillman 2y ago
XFS is a file system and ZFS is both a file system and a volume manager. ZFS also recently added RAIDZ expansion which I was really excited about.
avar 2y ago
If you know nothing about filesystems, then you're almost certainly limited by the underlying hardware, not the filesystem. And "performance" is meaningless in this context without specifying what you want to perform. Raw sequential read/write throughput, random metadata operations on directories containing millions of files? Something in between? > Are there ways to make a step-change in > overall performance somewhere? Yes, it's called O_DIRECT (or equivalent), and is very useful to those that actually need it. A general-use filesystem is inherently something that requires you to compromise on performance.
6SixTy 2y ago
A lot of NAND SSDs have controllers that pretty much take away a lot of control the rest of the system has over the actual flash cells. Pretty much any flash specific file systems are designed for embedded applications where there is no controller to perform wear leveling or TRIM operations. Pretty much most of the benefits that could be gained by now are at the application level maybe the CPU, not the OS nor filesystem.
evil-olive 2y ago
> Now that we have NAND storage, would we create filesystems differently? Are there ways to make a step-change in overall performance somewhere? you might enjoy Allan Jude's "Scaling ZFS for NVMe" talk [0] from 2022. the tl;dw is that modern NVMe drives have gotten so fast that filesystem developers are having to revisit assumptions that have been true for decades about which operations will be slow vs. fast. flash storage that exposed a SATA interface and pretended to be a superbly fast hard drive was an evolutionary improvement of sorts. NVMe is more of a revolutionary improvement, it does away with the spinning-rust assumptions. for example, SATA limits a device to one command queue with 32 commands; NVMe makes it almost unlimited (64k queues with up to 64k commands) [1] there is also a corner of the NVMe spec [2] that allows you to treat an SSD directly as key-value storage instead of block storage with a filesystem implementing a key-value layer on top of it. this is promising but AFAIK no mainstream filesystem is using it yet, and I'm not sure if support has even trickled down into consumer drives or if it's market-segmented off for datacenter use only. 0: https://www.youtube.com/watch?v=v8sl8gj9UnA https://www.youtube.com/watch?v=v8sl8gj9UnA 1: https://en.wikipedia.org/wiki/NVM_Express#Comparison_with_AHCI https://en.wikipedia.org/wiki/NVM_Express#Comparison_with_AH... 2: https://nvmexpress.org/specification/key-value-command-set-specification/ https://nvmexpress.org/specification/key-value-command-set-s...
chaostheo 2y ago
Still ZFS for stability ?? Pull the power plug while writing and you know what your stability is worth in reality. I would say say ZFS is for verify as long as you have a usv in front of your server otherwise with a poweroutage you should have a good backup and be able to do a full restore.

mtillman 2y ago

XFS is a file system and ZFS is both a file system and a volume manager. ZFS also recently added RAIDZ expansion which I was really excited about.

avar 2y ago

If you know nothing about filesystems, then you're almost certainly limited by the underlying hardware, not the filesystem. And "performance" is meaningless in this context without specifying what you want to perform. Raw sequential read/write throughput, random metadata operations on directories containing millions of files? Something in between? > Are there ways to make a step-change in > overall performance somewhere? Yes, it's called O_DIRECT (or equivalent), and is very useful to those that actually need it. A general-use filesystem is inherently something that requires you to compromise on performance.

6SixTy 2y ago

A lot of NAND SSDs have controllers that pretty much take away a lot of control the rest of the system has over the actual flash cells. Pretty much any flash specific file systems are designed for embedded applications where there is no controller to perform wear leveling or TRIM operations. Pretty much most of the benefits that could be gained by now are at the application level maybe the CPU, not the OS nor filesystem.

evil-olive 2y ago

> Now that we have NAND storage, would we create filesystems differently? Are there ways to make a step-change in overall performance somewhere? you might enjoy Allan Jude's "Scaling ZFS for NVMe" talk [0] from 2022. the tl;dw is that modern NVMe drives have gotten so fast that filesystem developers are having to revisit assumptions that have been true for decades about which operations will be slow vs. fast. flash storage that exposed a SATA interface and pretended to be a superbly fast hard drive was an evolutionary improvement of sorts. NVMe is more of a revolutionary improvement, it does away with the spinning-rust assumptions. for example, SATA limits a device to one command queue with 32 commands; NVMe makes it almost unlimited (64k queues with up to 64k commands) [1] there is also a corner of the NVMe spec [2] that allows you to treat an SSD directly as key-value storage instead of block storage with a filesystem implementing a key-value layer on top of it. this is promising but AFAIK no mainstream filesystem is using it yet, and I'm not sure if support has even trickled down into consumer drives or if it's market-segmented off for datacenter use only. 0: https://www.youtube.com/watch?v=v8sl8gj9UnA https://www.youtube.com/watch?v=v8sl8gj9UnA 1: https://en.wikipedia.org/wiki/NVM_Express#Comparison_with_AHCI https://en.wikipedia.org/wiki/NVM_Express#Comparison_with_AH... 2: https://nvmexpress.org/specification/key-value-command-set-specification/ https://nvmexpress.org/specification/key-value-command-set-s...

chaostheo 2y ago

Still ZFS for stability ?? Pull the power plug while writing and you know what your stability is worth in reality. I would say say ZFS is for verify as long as you have a usv in front of your server otherwise with a poweroutage you should have a good backup and be able to do a full restore.