4 ms·

ZFS don't really need ECC that much, just FUD. Most likely someone got bitten by faulty RAM, got corrupted data, which stopped after getting ECC RAM. => "ZFS w

by Elhana 10y ago

ZFS don't really need ECC that much, just FUD. Most likely someone got bitten by faulty RAM, got corrupted data, which stopped after getting ECC RAM. => "ZFS wants ECC" myth, but in reality you just need stable hardware. At the same time, if you are running like 30-40 TB pool, you probably want that ECC RAM, it will cost only small fraction of total storage box price and will save you from rebuilding a pool if shit happens.

wstrange 10y ago
It's not so much that ZFS needs ECC anymore than any other filesystem, but rather when you do have RAM errors, you are going to find out very quickly. Other filesystems will happily let you write corrupt data blocks...
- erkkie 10y ago
  See my other comment a bit up in the thread. ZFS does data checksumming which relies on the ram not lying to you.
  - the_why_of_y 10y ago
    But at least with ZFS checksums you have a statistical chance of detecting bad RAM because it will sometimes manifest as checksum errors, whereas with non-checksumming filesystems you just get silent data corruption.
__jal 10y ago
I don't know about anyone else, but when I'm seeking advice about software from people on the internet whom I don't know personally, I tend to weight the words of the authors of the software in question more highly than that of people who didn't write it. There are probably exceptions to that I'm currently forgetting. And the authors of ZFS, from the Sun days on, have been consistently and repeatedly saying that if you care about your data, use ECC. I personally don't get why people seem so resistant to using ECC. It isn't that much more expensive and we know these errors happen in around 8% of DIMMS[1]. It reminds me of people I know who refuse to buy a decent power supply and then complain about hardware failures because of shoddy power. Personally, paying a slight premium to avoid an almost one in 10 likelihood of silent data corruption seems like a no-brainer to me. But then, I care about my data. Edit: to be clear, ZFS will perform less dangerously with non-ECC ram than a filesystem that doesn't checksum with non-ECC, because it will detect Bad Things happening and tell you about it. ECC helps avoid the problems in the first place. [1] http://research.google.com/pubs/pub35162.html http://research.google.com/pubs/pub35162.html
- wtallis 10y ago
  ECC isn't that much more expensive, if you already have a processor and motherboard that give you that option. If you're trying to use ZFS on surplus consumer Intel equipment, ECC is a major expense. And aside from the cost issue, I don't think people are resistant to using ECC. You have probably just misinterpreted people who are justifiably shooting down ZFS/ECC scaremongering: Advising people to never use ZFS without ECC RAM is bad advice, because the advice should simply be to use ECC RAM if you want enterprise grade reliability, whether or not ZFS is part of the picture. Some people aren't in need of that level of reliability but can still benefit from ZFS, and they shouldn't be misled into thinking that ZFS has some particular need for ECC RAM.
  - __jal 10y ago
    I agree on the last part, and I think the particular weirdness here is a result of ZFS historically mentioning the value of ECC a lot more than other FSes. I think this probably makes people think that ZFS in some way depends on it more than others, rather than simply pointing out that there's a disturbingly high chance of encountering a problem without it that applies to everything. And sure, if you're building a frankenbox, ECC is probably not an option, or the sort of expense that takes it out of the frankenbox category. I do hope that folks wouldn't store important data on dodgy hardware, but that is about more than RAM, and also none of my business. I don't believe I'm misinterpreting people's reactions - I've witnessed people assert that ECC is a scam, waste of money and similar. Even after I point them to that link I posted above, they still seem to believe that It Won't Happen To Them. ("I don't have millions of machines.") In any case, I still find it frankly bizarre that people run the risk with important data. If you asked people if they wanted to buy a CPU that had an ~8% chance of undetectably lying to them, I'm pretty sure the vast majority would at least want to spring for the premium one that allows at the least detecting the lie.
    - wtallis 10y ago
      > I've witnessed people assert that ECC is a scam, waste of money and similar. Even after I point them to that link I posted above, they still seem to believe that It Won't Happen To Them. ("I don't have millions of machines.") If by your own citation 92% of DIMMs operate with zero errors per year, and a consumer machine has at most four DIMMs, then it is actually pretty likely that any given consumer machine will operate without RAM errors even without ECC. And when you multiply by the low probability that a DRAM error will cause catastrophic data loss, then it is very easy to come to a reasonable conclusion that ECC is not worth the expense.

by Elhana 10y ago

wstrange 10y ago

It's not so much that ZFS needs ECC anymore than any other filesystem, but rather when you do have RAM errors, you are going to find out very quickly. Other filesystems will happily let you write corrupt data blocks...

erkkie 10y ago
See my other comment a bit up in the thread. ZFS does data checksumming which relies on the ram not lying to you.
- the_why_of_y 10y ago
  But at least with ZFS checksums you have a statistical chance of detecting bad RAM because it will sometimes manifest as checksum errors, whereas with non-checksumming filesystems you just get silent data corruption.

__jal 10y ago

I don't know about anyone else, but when I'm seeking advice about software from people on the internet whom I don't know personally, I tend to weight the words of the authors of the software in question more highly than that of people who didn't write it. There are probably exceptions to that I'm currently forgetting. And the authors of ZFS, from the Sun days on, have been consistently and repeatedly saying that if you care about your data, use ECC. I personally don't get why people seem so resistant to using ECC. It isn't that much more expensive and we know these errors happen in around 8% of DIMMS[1]. It reminds me of people I know who refuse to buy a decent power supply and then complain about hardware failures because of shoddy power. Personally, paying a slight premium to avoid an almost one in 10 likelihood of silent data corruption seems like a no-brainer to me. But then, I care about my data. Edit: to be clear, ZFS will perform less dangerously with non-ECC ram than a filesystem that doesn't checksum with non-ECC, because it will detect Bad Things happening and tell you about it. ECC helps avoid the problems in the first place. [1] http://research.google.com/pubs/pub35162.html http://research.google.com/pubs/pub35162.html

wtallis 10y ago
ECC isn't that much more expensive, if you already have a processor and motherboard that give you that option. If you're trying to use ZFS on surplus consumer Intel equipment, ECC is a major expense. And aside from the cost issue, I don't think people are resistant to using ECC. You have probably just misinterpreted people who are justifiably shooting down ZFS/ECC scaremongering: Advising people to never use ZFS without ECC RAM is bad advice, because the advice should simply be to use ECC RAM if you want enterprise grade reliability, whether or not ZFS is part of the picture. Some people aren't in need of that level of reliability but can still benefit from ZFS, and they shouldn't be misled into thinking that ZFS has some particular need for ECC RAM.
- __jal 10y ago
  I agree on the last part, and I think the particular weirdness here is a result of ZFS historically mentioning the value of ECC a lot more than other FSes. I think this probably makes people think that ZFS in some way depends on it more than others, rather than simply pointing out that there's a disturbingly high chance of encountering a problem without it that applies to everything. And sure, if you're building a frankenbox, ECC is probably not an option, or the sort of expense that takes it out of the frankenbox category. I do hope that folks wouldn't store important data on dodgy hardware, but that is about more than RAM, and also none of my business. I don't believe I'm misinterpreting people's reactions - I've witnessed people assert that ECC is a scam, waste of money and similar. Even after I point them to that link I posted above, they still seem to believe that It Won't Happen To Them. ("I don't have millions of machines.") In any case, I still find it frankly bizarre that people run the risk with important data. If you asked people if they wanted to buy a CPU that had an ~8% chance of undetectably lying to them, I'm pretty sure the vast majority would at least want to spring for the premium one that allows at the least detecting the lie.
  - wtallis 10y ago
    > I've witnessed people assert that ECC is a scam, waste of money and similar. Even after I point them to that link I posted above, they still seem to believe that It Won't Happen To Them. ("I don't have millions of machines.") If by your own citation 92% of DIMMs operate with zero errors per year, and a consumer machine has at most four DIMMs, then it is actually pretty likely that any given consumer machine will operate without RAM errors even without ECC. And when you multiply by the low probability that a DRAM error will cause catastrophic data loss, then it is very easy to come to a reasonable conclusion that ECC is not worth the expense.