3 ms·
Disk is cheap, but how you attach the disks starts becoming an important cost component as you add more disks. :-( IOPS aren't cheap, and you will find out j
by raattgift 10y ago
Disk is cheap, but how you attach the disks starts becoming an important cost component as you add more disks. :-(
IOPS aren't cheap, and you will find out just how much that matters the moment you have to replace a disk in a pool with a lot of data in it. Resilver times of days can be pretty agonizing in a home setting.
I won't build a pool without at least double redundancy per rotating-drive vdev, meaning at least 3-way mirrors (zpool create foo mirror disk0 disk1 disk2 mirror disk3 disk4 disk5) or raidz2, as I have had to go to backups for failures during resilvers of raidz (never again!) and 2-way mirrors.
3-way mirrors conveniently provide high read IOPS. OpenZFS does a good job aggregating and scheduling writes to the point that writes are mainly choked by random reads. (Very full pools induce quasi-random reads while hunting for space in metaslabs, which is the main reason writes to very full pools perform poorly).
2-way mirrors are only really fine for vdevs that have negligible random-access penalty (and thus likely high IOPS), or where rebuilding an entire pool from backups takes time comparable to resilver and scrub times when dealing with a replacement leaf device that might fail during resilvering (you cannot know the data on the replacment device is OK until you have read it, which generally means scrub-after-resilver), or errors reading the surviving leaf device(s).