3 ms·
Surprised to not find 'ecc' on that page. I know it's not a guarantee of no-corruption, and ZFS without ECC is probably no more dangerous than any other file s
by Jedd 2y ago
Surprised to not find 'ecc' on that page.
I know it's not a guarantee of no-corruption, and ZFS without ECC is probably no more dangerous than any other file system without ECC, but if data corruption is a major concern for you, and you're building out a pretty hefty system like this, I can't imagine not using ECC.
Slow on-disk data corruption resulting from gradual and near-silent RAM failures may be like doing regular 3-2-1 backups -- you either mitigate against the problem because you've been stung previously, or you're in that blissful pre-sting phase of your life.
EDIT: I found TFA's link to the original build out - and happily they are in fact running a Xeon with ECC. Surprisingly it's a 16GB box (I thought ZFS was much hungrier on the RAM : disk ratio.) Obviously it hasn't helped for physical disk failures, but the success of the storage array owes a lot to this component.
- louwrentius 2y agoThe system is using ECC and I specifically - unrelated to ZFS - wanted to use ECC memory to reduce risk of data/fs corruption. I've also added 'ecc' to the original blog post to clarify. Edit: ZFS for home usage doesn't need a ton of RAM as far as I've learned. There is the 1 GB of RAM per 1TB of storage rule of thumb, but that was for a specific context. Maybe the ill-fated data deduplication feature, or was it just to sustain performance?
- rincebrain 2y agoIt was a handwavey rule of estimation for dedup, handwavey because dedup scales on number of records, which is going to vary wildly by recordsize.
- InvaderFizz 2y agoAdditionally unless it's changed in the last six years, you should pretend ZFS dedupe doesn't exist.
- rincebrain 2y agoNot in a stable release yet, but check out https://github.com/openzfs/zfs/discussions/15896 https://github.com/openzfs/zfs/discussions/15896 if you have a need for that.
- Jedd 2y agoThanks, and all good - it was my fault for not following the link in this story to your post about the actual build, before starting on my mini-rant. I'd heard the original ZFS memory estimations were somewhat exuberant, and recommendations had come down a lot since the early days, but I'd imagine given your usage pattern - powered on periodically - a performance hit for whatever operations you're doing during that time wouldn't be problematic. I used to use mdadm for software RAID, but for several years now my home boxes are all hardware RAID. LVM2 provides the other features I need, so I haven't really ever explored zfs as a replacement for both - though everyone I know that uses it, loves it.
- hinkley 2y agoAccidentally unplugged my raid 5 array and thought I damaged the raid card. Hours after boot I’d get problems. I glitched a RAM chip and the array was picking it up as disk corruption.
- Filligree 2y agoIt's difficult as a home user to find ECC memory, harder to make sure it actually works in your hardware configuration, and near-impossible to find ECC memory that doesn't require lower speeds than what you can get for $50 on amazon. I would very much like to put ECC memory in my home server, but I couldn't figure it out this generation. After four hours I decided I had better things to do with my time.
- Jedd 2y agoIndeed. I'd started to add an aside to the effect of 'ten years ago it was probably easier to go ECC'. I'll add it here instead. A decade ago if you wanted ECC your choice was basically Xeon, and all() Xeon motherboards would accept ECC. I agree that these days it's much more complex, since you are ineluctably going get sucked into the despair-spiral of trying to work out what combination of Ryzen + motherboard + ECC RAM will give you actual, demonstrable* ECC (with correction, not just detection).