3 ms·

A note about RAID arrays: Yes it is true that drives have become bigger but not faster. This causes RAID rebuilds to take longer. But I am not aware of any har

A note about RAID arrays: Yes it is true that drives have become bigger but not faster. This causes RAID rebuilds to take longer. But I am not aware of any hard evidence that this means that this increases the risk of a failed rebuild. There are more sectors that could be bad but you need to do a patrol read / scrub at least one month to detect them in advance. I think there are a lot of scares about RAID of which I really wonder how much of it is rooted in anekdote and folklore.

Craighead 7y ago
The thinking goes that the more total storage available the more unrecoverable bit errors occur
montjoy 7y ago
The main issue with big drives is that you are sitting in a degraded state for longer while you await the rebuild. That allows more time for another disk to fail while your rebuild is incomplete.
- zepearl 7y ago
  Right - more time in a degraded state and more stress on the remaining drives during the rebuild (because there will be more data to synchronize).
seized 7y ago
If you have a set of N drives all purchased at the same time, they are going to all have the same approximate bathtub curve and could start failing at the same time. I had five Seagate 7200.11s (terrible, shitty drives in many ways) in a RAIDZ1 pool. One failed, I started a resilver. Another started failing during that time. The rebuild slowed significantly due to the second failing drive. ZFS resilvered eventually then the second drive failed completely. That was at only 26k hours or so. I think a third started acting up shortly after but I was already replacing them all. These were only 500GB drives, and the chances aligned on two of them.... I only use RAIDZ2 now (and I don't use Seagates).
- zepearl 7y ago
  I had as well some Seagate drives failing... . (using now only HGST and WD drives - no experience with Toshiba) Question about ZFS and its RAIDZ: do you have any recommendation (personal experience, links, books, ...) concerning parameters/setup to be used when setting up a RAIDZ(1 and 2)? I'm new to ZFS and I already had to do a lot of tests when using ZFS only on 1 HDD until I finally managed to get good performance out of it (used by a "Clickhouse" database which itself writes data in a CoW-style => I had to raise the "recordsize" to 2MiB) and I imagine that with RAIDZ it can get more complicated? I would like to set up a RAIDZ for the database (again, "Clickhouse", which generates multi-GB files) and two more to be used as simple NAS (a main one and a backup, storing files of all sizes). I searched a lot and found some websites which were "ok", and bought as well 2 tiny books ( https://www.amazon.com/Introducing-ZFS-Linux-Understand-Storage-ebook/dp/B077QTFLY8/ref=sr_1_1?dchild=1&keywords=introducing+zfs+on+linux&qid=1587147735&sr=8-1 https://www.amazon.com/Introducing-ZFS-Linux-Understand-Stor... and https://www.amazon.com/ZFS-Linux-Administration-William-Speirs/dp/154462204X/ref=sr_1_1?dchild=1&keywords=zfs+internals+and+administration&qid=1587147772&sr=8-1 https://www.amazon.com/ZFS-Linux-Administration-William-Spei... ) but the books were mediocre and the informations I found in the web were sparse and a bit oldish. Cheers
  - seized 7y ago
    Servethehome.com has some subforums with a lot of info, particularly the Napp-it one which is a management UI for ZFS like FreeNAS is Reddit.com/r/ZFS Reddit.com/r/datahoarder I'll try to think of some others. Honestly the best way is to start tinkering and see how it works for you as it can depend heavily on what you're using it for. But yes there is a lot of FUD out there. One thing to beware of, RAIDZ1/2 is limited to the IOPS of one drive per VDEV... Can be quite limiting for some things.
    - zepearl 7y ago
      Thank you! Didn't know that website - it has a lot of interesting stuff. Ok, I'll then do some tests and will see how the RAIDZ behaves.

Craighead 7y ago

The thinking goes that the more total storage available the more unrecoverable bit errors occur

montjoy 7y ago

The main issue with big drives is that you are sitting in a degraded state for longer while you await the rebuild. That allows more time for another disk to fail while your rebuild is incomplete.

zepearl 7y ago
Right - more time in a degraded state and more stress on the remaining drives during the rebuild (because there will be more data to synchronize).

seized 7y ago

If you have a set of N drives all purchased at the same time, they are going to all have the same approximate bathtub curve and could start failing at the same time. I had five Seagate 7200.11s (terrible, shitty drives in many ways) in a RAIDZ1 pool. One failed, I started a resilver. Another started failing during that time. The rebuild slowed significantly due to the second failing drive. ZFS resilvered eventually then the second drive failed completely. That was at only 26k hours or so. I think a third started acting up shortly after but I was already replacing them all. These were only 500GB drives, and the chances aligned on two of them.... I only use RAIDZ2 now (and I don't use Seagates).

zepearl 7y ago
I had as well some Seagate drives failing... . (using now only HGST and WD drives - no experience with Toshiba) Question about ZFS and its RAIDZ: do you have any recommendation (personal experience, links, books, ...) concerning parameters/setup to be used when setting up a RAIDZ(1 and 2)? I'm new to ZFS and I already had to do a lot of tests when using ZFS only on 1 HDD until I finally managed to get good performance out of it (used by a "Clickhouse" database which itself writes data in a CoW-style => I had to raise the "recordsize" to 2MiB) and I imagine that with RAIDZ it can get more complicated? I would like to set up a RAIDZ for the database (again, "Clickhouse", which generates multi-GB files) and two more to be used as simple NAS (a main one and a backup, storing files of all sizes). I searched a lot and found some websites which were "ok", and bought as well 2 tiny books ( https://www.amazon.com/Introducing-ZFS-Linux-Understand-Storage-ebook/dp/B077QTFLY8/ref=sr_1_1?dchild=1&keywords=introducing+zfs+on+linux&qid=1587147735&sr=8-1 https://www.amazon.com/Introducing-ZFS-Linux-Understand-Stor... and https://www.amazon.com/ZFS-Linux-Administration-William-Speirs/dp/154462204X/ref=sr_1_1?dchild=1&keywords=zfs+internals+and+administration&qid=1587147772&sr=8-1 https://www.amazon.com/ZFS-Linux-Administration-William-Spei... ) but the books were mediocre and the informations I found in the web were sparse and a bit oldish. Cheers
- seized 7y ago
  Servethehome.com has some subforums with a lot of info, particularly the Napp-it one which is a management UI for ZFS like FreeNAS is Reddit.com/r/ZFS Reddit.com/r/datahoarder I'll try to think of some others. Honestly the best way is to start tinkering and see how it works for you as it can depend heavily on what you're using it for. But yes there is a lot of FUD out there. One thing to beware of, RAIDZ1/2 is limited to the IOPS of one drive per VDEV... Can be quite limiting for some things.
  - zepearl 7y ago
    Thank you! Didn't know that website - it has a lot of interesting stuff. Ok, I'll then do some tests and will see how the RAIDZ behaves.