3 ms·

> I don't know if this exists or not, but I'd like to try something like a fuse filesystem which can transparently copy a file to a fast scratch SSD when it is

> I don't know if this exists or not, but I'd like to try something like a fuse filesystem which can transparently copy a file to a fast scratch SSD when it is first accessed. ZFS has caching for writes (SLOG)[0][1] and reads (L2ARC),[2][3] which was introduced many years ago when HDDs were cheap and flash was still very, very expensive: * https://www.brendangregg.com/blog/2009-10-08/hybrid-storage-pool-top-speeds.html https://www.brendangregg.com/blog/2009-10-08/hybrid-storage-... [0] https://openzfs.github.io/openzfs-docs/man/master/7/zpoolconcepts.7.html#Intent_Log https://openzfs.github.io/openzfs-docs/man/master/7/zpoolcon... [1] https://openzfs.github.io/openzfs-docs/man/master/8/zpool-create.8.html#Example_6_:_Creating_a_ZFS_Pool_with_Mirrored_Separate_Intent_Logs https://openzfs.github.io/openzfs-docs/man/master/8/zpool-cr... [2] https://openzfs.github.io/openzfs-docs/man/master/7/zpoolconcepts.7.html#Cache_Devices https://openzfs.github.io/openzfs-docs/man/master/7/zpoolcon... [3] https://openzfs.github.io/openzfs-docs/man/master/8/zpool-add.8.html#Example_2_:_Adding_Cache_Devices_to_a_ZFS_Pool https://openzfs.github.io/openzfs-docs/man/master/8/zpool-ad...

aidenn0 2y ago
I should point out that the SLOG only caches synchronous writes, which are written twice with ZFS. Also, the L2ARC is great, but does still have RAM overhead. There are also useful tunables. I had a workload on a RAM limited machine where directory walking was common, but data reads were fairly random and a L2ARC configured for metadata only speed it up by a large amount.
- kaliszad 2y ago
  You can also have a special metadata only/ small files dedicated special VDEV. ZFS can pull of many tricks if you configure it well. Of course, the L2ARC is better, if you don't trust the caching device that much (e.g. only have a single SSD).
  - flemhans 2y ago
    You can also configure an L2ARC to only hold metadata, which is safer than a metadata device.
magicalhippo 2y ago
> ZFS has caching for writes Not really. It will accumulate synchronous writes into the ZIL, and you put the ZIL on a fast SLOG vdev. But it will only do so for a limited amount of time/space, and is not meant as a proper write-back cache but rather as a means to quickly service synchronous writes. By default asynchronous writes do not use the ZIL, and hence SLOG vdev at all. You can force it to, but that can also be a bad idea unless you have Optane drives as you're then bottlenecked by the ZIL/SLOG.
- telgareith 2y ago
  "Fast" is meaningless. A SLOG needs to have: good throughput and low latency at a queue depth of 1.

aidenn0 2y ago

I should point out that the SLOG only caches synchronous writes, which are written twice with ZFS. Also, the L2ARC is great, but does still have RAM overhead. There are also useful tunables. I had a workload on a RAM limited machine where directory walking was common, but data reads were fairly random and a L2ARC configured for metadata only speed it up by a large amount.

kaliszad 2y ago
You can also have a special metadata only/ small files dedicated special VDEV. ZFS can pull of many tricks if you configure it well. Of course, the L2ARC is better, if you don't trust the caching device that much (e.g. only have a single SSD).
- flemhans 2y ago
  You can also configure an L2ARC to only hold metadata, which is safer than a metadata device.

magicalhippo 2y ago

> ZFS has caching for writes Not really. It will accumulate synchronous writes into the ZIL, and you put the ZIL on a fast SLOG vdev. But it will only do so for a limited amount of time/space, and is not meant as a proper write-back cache but rather as a means to quickly service synchronous writes. By default asynchronous writes do not use the ZIL, and hence SLOG vdev at all. You can force it to, but that can also be a bad idea unless you have Optane drives as you're then bottlenecked by the ZIL/SLOG.

telgareith 2y ago
"Fast" is meaningless. A SLOG needs to have: good throughput and low latency at a queue depth of 1.