4 ms·

Replacing one disk at a time in a ZFS pool that's almost full is a bad idea, because all new writes will go to the sole new disk.

mbreese 7y ago
But that's the only way to make an existing pool bigger, isn't it? Replace each disk, one at a time, and then auto-expand the pool at the end. The usable space doesn't change until the pool expands at the end. I think you're thinking about adding a new vdev to an existing pool to make the pool larger. This is almost always a bad idea, because like you said, the writes all end up on the new vdev.
- gravypod 7y ago
  If you can have downtime: you could create a new pool, turn off writes to the old pool, snapshot the entire pool into the new pool using the built in snapshot & transfer mechanics, and then kill the old pool. If you're ok with write performance degradation & doing multiple resilvers than what the op did is fine too.
  - tracker1 7y ago
    That assumes you have enough extra spaces to do it and you didn't start full. Ex: just bought a used Dell R710 with 6 bays, it came with 2TB drives... plan is to play around a bit with those... but I don't have an extra, supported controller with enough slots (even if the cover is off) to populate it while I would migrate. In the end, it'll be full of shucked 8tb drives, but pretty sure I'll just start over at that point. My old nas' drives are 6yo in a decade old synology box, and my trust level is failing before any drives do.
    - gravypod 7y ago
      Something that's important to consider is if your drives can handle a resilver. Each resilver is essentially reading the entire disk (with lots of asterisks) while setting up a parallel zfs system to provision your disks and using the transfer I described will only read your systems fs once. You don't even need to use the same system. You can setups a different system, run the transfer, turn off the main system, swap the disks. It's a hastle but it's an option at your disposal.
      - tracker1 7y ago
        If I'm using Raid-Z2, isn't there a way to drop the old drive and add a new drive to take its' place?
radiowave 7y ago
I don't think that's correct for the case that dvdgsng is describing. IIRC, the RAIDZ vdev won't allow the use of any extra space until all of its drives have been replaced with bigger ones. What you're describing occurs when growing a ZFS pool by adding a new vdev, i.e. extra disks that sit alongside the existing ones. (Edit: Didn't realize that mbreese had already addressed this point in a different reply.)
- layoutIfNeeded 7y ago
  True, I didn't pay attention that OP was growing the vdev and not the pool.

mbreese 7y ago

But that's the only way to make an existing pool bigger, isn't it? Replace each disk, one at a time, and then auto-expand the pool at the end. The usable space doesn't change until the pool expands at the end. I think you're thinking about adding a new vdev to an existing pool to make the pool larger. This is almost always a bad idea, because like you said, the writes all end up on the new vdev.

gravypod 7y ago
If you can have downtime: you could create a new pool, turn off writes to the old pool, snapshot the entire pool into the new pool using the built in snapshot & transfer mechanics, and then kill the old pool. If you're ok with write performance degradation & doing multiple resilvers than what the op did is fine too.
- tracker1 7y ago
  That assumes you have enough extra spaces to do it and you didn't start full. Ex: just bought a used Dell R710 with 6 bays, it came with 2TB drives... plan is to play around a bit with those... but I don't have an extra, supported controller with enough slots (even if the cover is off) to populate it while I would migrate. In the end, it'll be full of shucked 8tb drives, but pretty sure I'll just start over at that point. My old nas' drives are 6yo in a decade old synology box, and my trust level is failing before any drives do.
  - gravypod 7y ago
    Something that's important to consider is if your drives can handle a resilver. Each resilver is essentially reading the entire disk (with lots of asterisks) while setting up a parallel zfs system to provision your disks and using the transfer I described will only read your systems fs once. You don't even need to use the same system. You can setups a different system, run the transfer, turn off the main system, swap the disks. It's a hastle but it's an option at your disposal.
    - tracker1 7y ago
      If I'm using Raid-Z2, isn't there a way to drop the old drive and add a new drive to take its' place?

radiowave 7y ago

I don't think that's correct for the case that dvdgsng is describing. IIRC, the RAIDZ vdev won't allow the use of any extra space until all of its drives have been replaced with bigger ones. What you're describing occurs when growing a ZFS pool by adding a new vdev, i.e. extra disks that sit alongside the existing ones. (Edit: Didn't realize that mbreese had already addressed this point in a different reply.)

layoutIfNeeded 7y ago
True, I didn't pay attention that OP was growing the vdev and not the pool.