2 ms·

Only time I've lost data with zfs was when i was first doing something supremely stupid, mostly as a dare to see if it would work. I had recently upgraded my p

Only time I've lost data with zfs was when i was first doing something supremely stupid, mostly as a dare to see if it would work. I had recently upgraded my pool and had a bunch of old disks lying around so i fired them up in a new pool, 6x 4TB and 6x 8TB disks. I took the 4TB disks and set them in in a raid0 to get effectively 9x 8TB disks. Apparently doing this will work but you MUST make sure you properly wipe all of the newly raided disks of their old ZFS data or you can end up with ZFS trying to check that pool and breaking the mdadm superblocks. After I properly zeroed the 4TB disks it's been working fine for weeks. This let me set it up a pool in raidz3 with them all so i can loose any of the much older 4tb disks or up to 3 of the 8tb ones. I don't use it for anything that's absolutely critical to store, just random bulk data that i don't want to worry that much about, but it's been reliable enough since then.

zeotroph 4y ago
Is there a zfs native way to create raid0 disks now? On Linux I usually set up a linear mapping with dmsetup, and then point zfs to that. But this hides the native disks from zfs. I've done that a couple times to make a bunch of mismatched disk fit into zfs's requirement that all are of the same size. The ability to work with a mix of disk sizes is one advantage of btrfs - if you remember to be extra careful on a disk failure and bad shutdown. Plus the ability to dynamically grow the array, but there is work to improve that in the zfs side as well.
- mgerdts 4y ago
  > Is there a zfs native way to create raid0 disks now? zpool create swimming disk1 disk2 … That has worked for over 15 years.
  - derkades 4y ago
    This does not create a RAID 0 / stripe, but it is close.
    - mgerdts 4y ago
      RAID0 could be striping which would require all devices to be the same size or it could be concatenation which leaves it to some other layer to balance load across drives. What I suggested allows the use of the full capacity of the drives with weighted data placement based on available space on the drives. What I suggested is not really RAID0. raidz is not really RAID5. ZFS intentionally improves on these for performance and reliability reasons.
  - zeotroph 4y ago
    Indeed, it is less raid0 but creating a raid0 and then building a raidz on top of it. Say you have `1, 1, 2, 2` sized devices. A raidz requires them all to be equally sized which can be done by first adding `1 + 1 = 2` and then creating a zpool with `[1+1], 2, 2`. Can this 1 + 1 step be done within zfs? Hypothetically this would be done by creating a raid0 pool and then a raidz on top via `zfs create tank raidz (raid0 a1 b1) c2 d2`, or first running `zfs create ab1 a1 ab` and use that in the next `zfs create ab1 raidz ab1 c2 d2`.
- simcop2387 4y ago
  I believe you can accomplish this with partitioning the disks with sizes that match the smallest disk too for zfs. You still have all the same reliability issues though and there's probably some kimd of performance hit too. Though yoy could then likely set copies=2 or something to add some redundancy to bad sectors at least and get a little better there than the linear mapping with dm or lvm2 on linux. You should the. Also be able to grow the array but like the linear mapping it's going to be unbalanced as to where free space is. My personal preference in this kind of situation is to still just use lvm2 to jbod the disks together if I;m fine with the risks of losing it all anyway, and just setup a good backup strategy. More because I'm familiar with it and all the tools than other setups. I think zfs might give me more early warning signs of a disk with an issue but the performance hit isn't always worth it to me there. I've got an 8tb raid0 nvme ssd array (4x2tb) setup to give me some blazing fast local storage for a few key VMs thst I just backup daily to the zfs storage. I can get 7.5GB/s reads and 6.7GB/s writes to that array doing that and using an lvm-thin pool so I can nicely snapshot during backups.
- wil421 4y ago
  You can stripe disks (raid0) in ZFS but you should not be using any hardware raid solution, motherboard or pcie card. Host bus adapters are preferred, anything with direct access to the disks.

zeotroph 4y ago

Is there a zfs native way to create raid0 disks now? On Linux I usually set up a linear mapping with dmsetup, and then point zfs to that. But this hides the native disks from zfs. I've done that a couple times to make a bunch of mismatched disk fit into zfs's requirement that all are of the same size. The ability to work with a mix of disk sizes is one advantage of btrfs - if you remember to be extra careful on a disk failure and bad shutdown. Plus the ability to dynamically grow the array, but there is work to improve that in the zfs side as well.

mgerdts 4y ago
> Is there a zfs native way to create raid0 disks now? zpool create swimming disk1 disk2 … That has worked for over 15 years.
- derkades 4y ago
  This does not create a RAID 0 / stripe, but it is close.
  - mgerdts 4y ago
    RAID0 could be striping which would require all devices to be the same size or it could be concatenation which leaves it to some other layer to balance load across drives. What I suggested allows the use of the full capacity of the drives with weighted data placement based on available space on the drives. What I suggested is not really RAID0. raidz is not really RAID5. ZFS intentionally improves on these for performance and reliability reasons.
- zeotroph 4y ago
  Indeed, it is less raid0 but creating a raid0 and then building a raidz on top of it. Say you have `1, 1, 2, 2` sized devices. A raidz requires them all to be equally sized which can be done by first adding `1 + 1 = 2` and then creating a zpool with `[1+1], 2, 2`. Can this 1 + 1 step be done within zfs? Hypothetically this would be done by creating a raid0 pool and then a raidz on top via `zfs create tank raidz (raid0 a1 b1) c2 d2`, or first running `zfs create ab1 a1 ab` and use that in the next `zfs create ab1 raidz ab1 c2 d2`.
simcop2387 4y ago
I believe you can accomplish this with partitioning the disks with sizes that match the smallest disk too for zfs. You still have all the same reliability issues though and there's probably some kimd of performance hit too. Though yoy could then likely set copies=2 or something to add some redundancy to bad sectors at least and get a little better there than the linear mapping with dm or lvm2 on linux. You should the. Also be able to grow the array but like the linear mapping it's going to be unbalanced as to where free space is. My personal preference in this kind of situation is to still just use lvm2 to jbod the disks together if I;m fine with the risks of losing it all anyway, and just setup a good backup strategy. More because I'm familiar with it and all the tools than other setups. I think zfs might give me more early warning signs of a disk with an issue but the performance hit isn't always worth it to me there. I've got an 8tb raid0 nvme ssd array (4x2tb) setup to give me some blazing fast local storage for a few key VMs thst I just backup daily to the zfs storage. I can get 7.5GB/s reads and 6.7GB/s writes to that array doing that and using an lvm-thin pool so I can nicely snapshot during backups.
wil421 4y ago
You can stripe disks (raid0) in ZFS but you should not be using any hardware raid solution, motherboard or pcie card. Host bus adapters are preferred, anything with direct access to the disks.