8 ms·
I don't agree. And if one of your disks failed, and age was a factor… you’re going to be sweating bullets wondering if another will fail before your resilver c
by tjoff 6y ago
I don't agree.
And if one of your disks failed, and age was a factor… you’re going to be sweating bullets wondering if another will fail before your resilver completes.
So every every single time you loose any drive in a mirror setup you risk all the data on all drives in the entire pool. I sure do hope you aren't on vacation and/or have to order a drive online.
It all depends on your use case. For me, raidz3 wins easily. Performance during resilvering is not something most users would suffer that much from anyway either. And that 8-drive recommendation comes from many factors and don't apply to most home-users anyway. Not that you should go overboard with it but it will easily pay for fast cache or whatever else you might want.
If performance was the goal you would not be using spinning rust anyway, and if you are still limited by a single gbit then don't even think about it (but don't over-utilize your pool).
But there are still lots of potential ways for your data to die, and you still need to back up your pool. Period. PERIOD!
Of course! Yet for home users there is not a single reasonable way to do it if you have decently sized pool. There is bound to be sacrifices on what you choose to backup.
Except for maybe another pool. Which is going to hurt since ZFS with buying everything up front is VASTLY more expensive than RAID where you can grow the array as needed - under the assumption that the storage needs grow slowly, which they typically do for home users.
As a ZFS user and fanboy the temptation of a dual raid6 setup is tempting. But I can't compromise on the filesystem so in the end I've compromised on backups instead. Likely not the smartest move considering how rare bitrot is, I am very well aware.
For many drives I'd go for raidz3. And for a 4-drive NAS I'd go with raidz2 rather than a mirror setup for the ease of mind.
For fast SSD pools I'd go with mirrors. Much easier to backup the entire pool as well.
- necheffa 6y agoRepeat after me: RAID is not a backup solution, its an uptime solution.
- tjoff 6y agoRAID is not a backup solution, its an uptime solution. I know. Doesn't change the fact there exist no sensible way to backup a decently sized pool in a home-user scenario. Now we might disagree on what is sensible. And some things might not need a backup.
- cortesoft 6y agoIsn’t the backup solution to always get two of whatever you build? A primary raid backed up to a second pool?
- Cyph0n 6y agoNot everyone has that kind of cash lying around, especially if the pool is large and the data is not critical.
- hedora 6y agoTypically three. Two local, one remote. Consider not RAID protecting the local backup.
- dspillett 6y agoYou need to do more than just keep a backup of current state, if you want to deal with problems other than complete failure/destruction of the primary. What about accidental deletion and incorrect editing (done too long ago for "undo" to work)?
- xoa 6y ago>What about accidental deletion and incorrect editing (done too long ago for "undo" to work)? Eh? We're talking ZFS here. Built-in atomic low cost snapshots are part of the core feature set. If we're talking using a pool and backing up another pool it's a reasonable baseline assumption that one would be using either their own or one of the many mature scripts to have regular automatic snapshots and retention schedule. Ie., once every 10 min for a day, once an hour for a week or two, once a week for a year, and monthly thereafter (or whatever fits your use case), along with preservation of any manual custom snapshots. I mean, it's right there, for free.
- dspillett 6y agoFair point, but the comment I replied to did not mention configuring anything like that, and while it is all there for the taking it isn't automatically configured.
- corndoge 6y agoNothing in gp's post even remotely indicated that they thought it was
- bluGill 6y agoFalse. There are too many different reasons for a backup to make that statement, and for most people RAID and NAS are combined into one. There is backup because a disks are unreliable and fail randomly. RAID is designed to solve this. There is backup because you accidentally deleted a file. There is backup because ransomware encrypted your files. RAID with snapshots solve these (arguably better than most other backup systems). There is backup because your main computers (phone) might break. NAS systems are a good place to backup these. There is backup because your house burned down [or otherwise is destroyed]. This is the only one that RAID doesn't solve. (even then you can do offsite NAS) What risks do you need to mitigate? Once you have that list you can decide what you mitigations to apply. RAID is a very useful piece of this and so saying RAID is not backup is not helpful and may harm people from making useful backups because they let the search for perfection delay them.
- djmobley 6y agoRAID is not a backup. It really is as simple as that, and it is helpful to remind people so they do not suffer the harms likely to result from mistakenly believing that RAID is a backup.
- Fnoord 6y agoIf you got disks from the same brand, and same batch, chances are they will fail at the same time before a resilver.
- mindslight 6y agoThis is true in a vacuum, but I feel it misses the mark having run my own personal array for 20+ years. For the most part, individuals have very little mission critical data (unless they're videographers or professional photographers). Ergo, most data on personal arrays is going to be non-critical - eg VM images, old drive images, build trees, downloaded datasets, torrented media. Does it make sense to double your cost (/halve your capacity) by backing up the whole thing, especially when the backup is likely to be more spinning rust? I've recently chosen to do so (while also backing up the critical data several different ways), but haven't always. I can understand people who still choose otherwise. Either way my main concern is a single raidz3 and I'm going to have a bad week if I lose that, independent of what can be rebuilt. In general, personal users of ZFS are very different than institutional users of ZFS. Hopefully not too divergent, so those raidz code paths still receive lots of testing and support. But you need to always keep this in mind when reading any advice about ZFS.
- jiveturkey 6y ago> Which is going to hurt since ZFS with buying everything up front is VASTLY more expensive than RAID where you can grow the array as needed - under the assumption that the storage needs grow slowly, which they typically do for home users. Not with mirrored vdevs, which you can expand pair by pair. I'm not arguing for the article, just pointing out your error. The article is very flawed and is bad advice. Not because it's bad per se, but the reasoning fails at basic mathematical analysis. There certainly are valid reasons to use mirrored vdevs. One might be, so you can more easily expand your pool incrementally. Anyway, from personal experience, my thought is that home users should give up on the idea of "incremental expansion" anyway. Double up each time. I'm surprised that you want to say home users aren't going to use 8 drive setups, while at the same time decrying mirrors in favor of parity. If you have a low drive count, the storage efficiency of parity puts you close to that of mirrors anyway. raidz2 with 4 drives as you suggest makes almost no sense vs mirrors.
- tjoff 6y agoGood point. Though the "internet wisdom" nowadays is to not add vdevs to a pool that has already seen use (in this scenario there will be a lot of use) since the pool will be very unbalanced and there is no way to rebalance it. Also negating many of the benefits of mirrors over raidz. But that advice might not be relevant for the use-cases discussed here, something to keep in mind. I meant that home-users don't have to be limited by 8-drive setups.
- z3t4 6y agoYou can add mirrors to increase the pool size. More RAM will speed up the pool, but cache drives also works really well. And you can have a hot spare.
- aidenn0 6y ago> Of course! Yet for home users there is not a single reasonable way to do it if you have decently sized pool. There is bound to be sacrifices on what you choose to backup. It's $130 for an 8TB USB drive at Best Buy. If you don't mirror your backup, that's $16/TB cost, which is quite reasonable.
- mulmen 6y agoWhat’s wrong with rsync.net or tarsnap for backups?
- ubercow13 6y agoThey are extremely expensive.
- mulmen 6y agoWow yeah I just did the math again and they are! Tarnsap is 10x S3 prices? I know the service does a lot but I thought it was a small margin on top of plain object storage.
- Dylan16807 6y agoIt's hard to price a backup service per byte when all the value comes from software. At the small end there's barely any profit, and at the big end the costs get ridiculous.
- mulmen 6y agoYeah I don’t begin to understand the economics. I was just surprised at the difference. I naively thought storage is cheap so backups are too.
- sigstoat 6y ago> Except for maybe another pool. Which is going to hurt since ZFS with buying everything up front is VASTLY more expensive than RAID where you can grow the array as needed - under the assumption that the storage needs grow slowly, which they typically do for home users. your backup pool doesn't have to match your working pool. it certainly doesn't have to be the same size. it just has to be larger than the data you've got. so you can build a big 64TB pool, and back it up to a set of external USB drives which expand as your actual data expands. similarly you can turn on a bit more compression for the backup pool than you might want on the main one.
- White_Wolf 6y agotbh for my 10 drive home setup I use a Z3 setup. Had dead drives but never more than 1 at a time. I do check them weekly and if they show signs of issues I replace them on the spot though. I do rsync to a second unit in a diff location but (so far) never had to do a restore.
- 67868018 6y agoYour mirrors can be more than 2 wide, you can have automatic hot standbys, you can make each mirror its own zpool so loss of some drives doesn’t lose the entire pool. Actually losing a mirror vdev doesn’t lose the whole pool anymore for several years now. I’ve recovered data off a zpool where I lost one of three mirrored vdevs. It’s not pretty, but your data is still there. Any files on the missing drives are just 0 bytes.
- jjav 6y ago> So every every single time you loose any drive in a mirror setup you risk all the data on all drives in the entire pool. Not true. If you lost all the drives in a mirror vdev, you lost those files not the entire pool. More importantly, mirrors are not limited to two drives. You can have any number. I use mirror vdevs with 4 drives.