4 ms·

The thing I'm struggling with is 4K sector support. It's horribly inefficient with ZFS. RAIDZ2 wastes a ton of space when pool is made with ashift=12. And every

The thing I'm struggling with is 4K sector support. It's horribly inefficient with ZFS. RAIDZ2 wastes a ton of space when pool is made with ashift=12. And everybody knows 512e on AF disks is horribly slow...so ZFS is either very slow or wastes 10% of total space...Or both (ZVOL :D) According to some bug reports, nobody has touched this since 2011...

iooi 9y ago
Can you elaborate on how ZFS wastes 10% of total space? I recently set up a ZFS volume using 12x4TB drives using RAID-Z2, so I expected 40TB of usable space, or ~36.3TiB. However, I only see 32TiB of usable space on the volume. I always wondered why that was so, never figured it out..
- gulikoza 9y ago
  There's ton of sites when you google ashift=12, http://louwrentius.com/zfs-performance-and-capacity-impact-of-ashift9-on-4k-sector-drives.html http://louwrentius.com/zfs-performance-and-capacity-impact-o... or https://github.com/zfsonlinux/zfs/issues/548 https://github.com/zfsonlinux/zfs/issues/548 for instance. Basically, ashift=12 increases ZFS block size to 4K. Metadata use full blocks that would be 512b on ashift=9 but are now 4K (due to ashift=12). It wastes at least 3.5Kb more than normal 512 byte blocks for each block that is not filled entirely.
  - iooi 9y ago
    Thanks!
  - sfoskett 9y ago
    I didn't even consider this. Thanks for the explanation! Makes tons of sense! Incidentally, many modern filesystems (including NTFS) store very small files in the FAT rather than taking up a whole block for this very reason!
    - chungy 9y ago
      cringes at "in the FAT" instead of "MFT".... ZFS has this same feature, however, as long as feature@embedded_data=enabled ;)
      - sfoskett 9y ago
        I guess I'm an old storage guy. I call it the FAT on everything! :-)
        takeda 9y ago
        Yeah, different filesystem calls things differently FAT, MFT, inodes. Best is to just call it metadata :)
  - secabeen 9y ago
    For new data, you can get back most of that lost space by setting the recordsize to 1M. You won't necessarily see the improvement in your df and zfs list commands, as they assume smaller max recordsize, but your overhead should drop significantly.

iooi 9y ago

Can you elaborate on how ZFS wastes 10% of total space? I recently set up a ZFS volume using 12x4TB drives using RAID-Z2, so I expected 40TB of usable space, or ~36.3TiB. However, I only see 32TiB of usable space on the volume. I always wondered why that was so, never figured it out..

gulikoza 9y ago
There's ton of sites when you google ashift=12, http://louwrentius.com/zfs-performance-and-capacity-impact-of-ashift9-on-4k-sector-drives.html http://louwrentius.com/zfs-performance-and-capacity-impact-o... or https://github.com/zfsonlinux/zfs/issues/548 https://github.com/zfsonlinux/zfs/issues/548 for instance. Basically, ashift=12 increases ZFS block size to 4K. Metadata use full blocks that would be 512b on ashift=9 but are now 4K (due to ashift=12). It wastes at least 3.5Kb more than normal 512 byte blocks for each block that is not filled entirely.
- iooi 9y ago
  Thanks!
- sfoskett 9y ago
  I didn't even consider this. Thanks for the explanation! Makes tons of sense! Incidentally, many modern filesystems (including NTFS) store very small files in the FAT rather than taking up a whole block for this very reason!
  - chungy 9y ago
    cringes at "in the FAT" instead of "MFT".... ZFS has this same feature, however, as long as feature@embedded_data=enabled ;)
    - sfoskett 9y ago
      I guess I'm an old storage guy. I call it the FAT on everything! :-)
      - takeda 9y ago
        Yeah, different filesystem calls things differently FAT, MFT, inodes. Best is to just call it metadata :)
- secabeen 9y ago
  For new data, you can get back most of that lost space by setting the recordsize to 1M. You won't necessarily see the improvement in your df and zfs list commands, as they assume smaller max recordsize, but your overhead should drop significantly.