3 ms·
> Average write operation duration decreased by 50% on our fullest machines This is probably not a fair comparison. On the existing lz4 machines, the zfs pool
by throwdbaaway 5y ago
> Average write operation duration decreased by 50% on our fullest machines
This is probably not a fair comparison.
On the existing lz4 machines, the zfs pools are already badly fragmented, making it difficult to find regular blocks: https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSGangBloc https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSGangBloc...
On the new zstd machines, the zfs pools are still pristine, after being restored from backup.
If we want to isolate the effects of migrating from lz4 to zstd, we also need some new lz4 machines for comparison.
- abrookewood 5y agoThat's a great point. Potentially they could implement regular replacement of nodes to obtain the benefits of de-fragmented pools.
- throwdbaaway 5y agoIndeed. It looks like they went from ZFS 0.8.x (lz4) to 2.x (zstandard) in one go. That would be too yolo for my taste, but the guys were under time pressure due to the rather sudden performance degradation. Nodes replacement would probably buy some time, to allow the upgrade to be done in 2 phases.
- Ford__Prefect 5y agoFor read frag on ssd shouldn matter, and for write since zfs is tree struct the allocator would create few more nodes and links but no big deal ?
- ericbarrett 5y agoLink is 404 for me.
- phaer 5y agoThe link seems to be truncated above: https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSGangBlocks https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSGangBloc...