3 ms·

ZFS forces you to choose a drive size a priori and live with it for the life of the system. Thanks but no thanks. I'll stick with OpenStack Swift where I can le

ZFS forces you to choose a drive size a priori and live with it for the life of the system. Thanks but no thanks. I'll stick with OpenStack Swift where I can leverage new larger disks and have my uncoupled highly available storage to boot.

chad_c 10y ago
This is not true. Read up on mirrored vdevs. http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/ http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-...
- gsmethells 10y ago
  50% storage efficiency and 87.5% survival is not an option for us. We store medical images by the petabyte and drives fail much too fast at scale.
  - gsmethells 10y ago
    Especially with the long resilver times necessary to achieve a good ROI from the investment.
  - notmyname 10y ago
    It's great to hear how you are using Swift. I work on that project; feel free to reach out if you have any questions. Contact info is in my profile.
Annatar 10y ago
That is not correct: one can easily increase the size of the pool by adding larger devices. As soon as the last device is replaced, the pool instantaneously possesses the upgraded capacity. No complicated grow commands, no filesystem expansions. It JustWorks(SM). zpool set autoreplace=on pool_name zpool set autoexpand=on pool_name properties must be set on the pool for this to work, either before or after the pool upgrade. Note that older versions of zpool(1M), like for example zpool version 15, do not have the autoexpand property. What cannot be done is a reduction in pool's capacity, but that does not come into these tales. There is no "apriori size determination", as zpool(1M) uses the entire disks or logical units, and ZFS filesystems use only as much capacity as they require, and no more, leading to extremely efficient disk space utilization.
- gsmethells 10y ago
  That is a lot of work compared to just adding a disk and rebalancing. No need to replace every disk in a vdev just to increase disk utilization to the entire disk when using Swift. No fear of losing a pool when losing a vdev.
  - Annatar 10y ago
    "Increase disk utilization when using Swift"? What you wrote makes no sense. There is never any fear of losing a pool when losing a vdev, as ZFS will let one simulate the entire thing by letting one use files as disks: % mkfile -v 128m /var/tmp/d0 /var/tmp/d0 134217728 bytes % mkfile -v 128m /var/tmp/d1 /var/tmp/d1 134217728 bytes % sudo zpool create testpool0 mirror /var/tmp/d0 /var/tmp/d1 Password: % zpool status pool: testpool0 state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM testpool0 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 /var/tmp/d0 ONLINE 0 0 0 /var/tmp/d1 ONLINE 0 0 0 errors: No known data errors % zfs list NAME USED AVAIL REFER MOUNTPOINT testpool0 988K 79.0M 952K /Volumes/testpool0 % sudo zpool set autoexpand=on testpool0 % mkfile -v 192m /var/tmp/d2 /var/tmp/d2 201326592 bytes % mkfile -v 192m /var/tmp/d3 /var/tmp/d3 201326592 bytes % sudo zpool replace testpool0 /var/tmp/d0 /var/tmp/d2 % sudo zpool replace testpool0 /var/tmp/d1 /var/tmp/d3 % zfs list NAME USED AVAIL REFER MOUNTPOINT testpool0 1008K 143M 952K /Volumes/testpool0 % sudo zpool destroy testpool0 Running process: '/usr/sbin/diskutil' 'unmount' '/Volumes/testpool0' Unmount successful for /Volumes/testpool0 % rm /var/tmp/d[0-9]
    - gsmethells 10y ago
      Did you not read the other guys link above? http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/ http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-... It says "Fault tolerance / degraded performance Be careful here. Keep in mind that if any single vdev fails, the entire pool fails with it. There is no fault tolerance at the pool level, only at the individual vdev level!"

chad_c 10y ago

This is not true. Read up on mirrored vdevs. http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/ http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-...

gsmethells 10y ago
50% storage efficiency and 87.5% survival is not an option for us. We store medical images by the petabyte and drives fail much too fast at scale.
- gsmethells 10y ago
  Especially with the long resilver times necessary to achieve a good ROI from the investment.
- notmyname 10y ago
  It's great to hear how you are using Swift. I work on that project; feel free to reach out if you have any questions. Contact info is in my profile.

Annatar 10y ago

That is not correct: one can easily increase the size of the pool by adding larger devices. As soon as the last device is replaced, the pool instantaneously possesses the upgraded capacity. No complicated grow commands, no filesystem expansions. It JustWorks(SM). zpool set autoreplace=on pool_name zpool set autoexpand=on pool_name properties must be set on the pool for this to work, either before or after the pool upgrade. Note that older versions of zpool(1M), like for example zpool version 15, do not have the autoexpand property. What cannot be done is a reduction in pool's capacity, but that does not come into these tales. There is no "apriori size determination", as zpool(1M) uses the entire disks or logical units, and ZFS filesystems use only as much capacity as they require, and no more, leading to extremely efficient disk space utilization.

gsmethells 10y ago
That is a lot of work compared to just adding a disk and rebalancing. No need to replace every disk in a vdev just to increase disk utilization to the entire disk when using Swift. No fear of losing a pool when losing a vdev.
- Annatar 10y ago
  "Increase disk utilization when using Swift"? What you wrote makes no sense. There is never any fear of losing a pool when losing a vdev, as ZFS will let one simulate the entire thing by letting one use files as disks: % mkfile -v 128m /var/tmp/d0 /var/tmp/d0 134217728 bytes % mkfile -v 128m /var/tmp/d1 /var/tmp/d1 134217728 bytes % sudo zpool create testpool0 mirror /var/tmp/d0 /var/tmp/d1 Password: % zpool status pool: testpool0 state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM testpool0 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 /var/tmp/d0 ONLINE 0 0 0 /var/tmp/d1 ONLINE 0 0 0 errors: No known data errors % zfs list NAME USED AVAIL REFER MOUNTPOINT testpool0 988K 79.0M 952K /Volumes/testpool0 % sudo zpool set autoexpand=on testpool0 % mkfile -v 192m /var/tmp/d2 /var/tmp/d2 201326592 bytes % mkfile -v 192m /var/tmp/d3 /var/tmp/d3 201326592 bytes % sudo zpool replace testpool0 /var/tmp/d0 /var/tmp/d2 % sudo zpool replace testpool0 /var/tmp/d1 /var/tmp/d3 % zfs list NAME USED AVAIL REFER MOUNTPOINT testpool0 1008K 143M 952K /Volumes/testpool0 % sudo zpool destroy testpool0 Running process: '/usr/sbin/diskutil' 'unmount' '/Volumes/testpool0' Unmount successful for /Volumes/testpool0 % rm /var/tmp/d[0-9]
  - gsmethells 10y ago
    Did you not read the other guys link above? http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/ http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-... It says "Fault tolerance / degraded performance Be careful here. Keep in mind that if any single vdev fails, the entire pool fails with it. There is no fault tolerance at the pool level, only at the individual vdev level!"