4 ms·

The same for ZFS; there's provisioning to make a "zil" device - ZFS Intent Log, basically the journal. ZFS is a little nicer in that this journal is explicitly

The same for ZFS; there's provisioning to make a "zil" device - ZFS Intent Log, basically the journal. ZFS is a little nicer in that this journal is explicitly disposable - If you lose your ZIL device, you lose any writes since it's horizon, but you don't lose the whole array. The next step up is building a "metadata" device, which stores the filesystem metadata but not data. This is dangerous in the way the ext4 journal is; lose the metadata, and you lose everything. Both are massive speedups. When doing big writes, a bunch of spinning rust can't achieve full throughput without a SSD ZIL. My 8+2 array can write nearly two gigabits, but it's abysmal (roughly the speed of a single drive) without a ZIL. Likewise, a metadata device can make the whole filesystem feel as snappy as SSD, but it's unnecessary if you have enough cache space; ZFS prefers it, so if your metadata fits into your cache SSD, most of it will stay loaded

Szpadel 1y ago
I just want to mention that ZIL is just to speed up sync writes, as it ends syscall when data are written to ZIL, but might be still in progress on slower storage. ZIL is also basically write only storage, therefore sad without very significant over provisioning will die quickly (you only read from ZIL after unclean shutdown) if you don't really case about latest version of file (risk of loosing recent chances is acceptable) you might set sync=disabled for that dataset and you can have great performance without ZIL
- magicalhippo 1y ago
  Minor nitpick, your post is primarily talking about SLOG, separate intent log. The pool always has a ZIL, but you can put it on a separate device, or decices, with SLOG[1]. [1]: https://www.truenas.com/docs/references/zilandslog/ https://www.truenas.com/docs/references/zilandslog/
JonChesterfield 1y ago
There's a configuration option that amounts to putting a directory (or maybe a volume) entirely into the metadata drive. It's been a long time since I set that up, but the home storage has spinning rust plus a raid 1 of crucial ssd (sata! But ones with a capacitor to hopefully handle writes after power loss), where the directory I care about performance for lives on the ssd subarray. Still presents as one blob of storage. Metadata on the ssd too, probably no ZIL but could be wrong about that. Made ls a lot more reasonable. Thinking about it that system must be trundling towards expected death, it might be a decade old now.

Szpadel 1y ago

I just want to mention that ZIL is just to speed up sync writes, as it ends syscall when data are written to ZIL, but might be still in progress on slower storage. ZIL is also basically write only storage, therefore sad without very significant over provisioning will die quickly (you only read from ZIL after unclean shutdown) if you don't really case about latest version of file (risk of loosing recent chances is acceptable) you might set sync=disabled for that dataset and you can have great performance without ZIL

magicalhippo 1y ago
Minor nitpick, your post is primarily talking about SLOG, separate intent log. The pool always has a ZIL, but you can put it on a separate device, or decices, with SLOG[1]. [1]: https://www.truenas.com/docs/references/zilandslog/ https://www.truenas.com/docs/references/zilandslog/

JonChesterfield 1y ago

There's a configuration option that amounts to putting a directory (or maybe a volume) entirely into the metadata drive. It's been a long time since I set that up, but the home storage has spinning rust plus a raid 1 of crucial ssd (sata! But ones with a capacitor to hopefully handle writes after power loss), where the directory I care about performance for lives on the ssd subarray. Still presents as one blob of storage. Metadata on the ssd too, probably no ZIL but could be wrong about that. Made ls a lot more reasonable. Thinking about it that system must be trundling towards expected death, it might be a decade old now.