7 ms·

There is nothing special about storage that makes the absence of ECC in a computer any more of a bad idea than it already is. Building machines without ECC is a

by ryao 10y ago

There is nothing special about storage that makes the absence of ECC in a computer any more of a bad idea than it already is. Building machines without ECC is a bad idea.

tracker1 10y ago
When writing redundant (parity data) across multiple storage devices, you're increasing the chance of error without ECC. While there's still a chance/risk with single drive, the effects of such error are not as significant.
- empthought 10y ago
  The effects are exactly as significant either way: corrupt data is written to disk. Whether the corrupt data is written one time or several times is immaterial.
  - jonathankoren 10y ago
    May I introduce you to error correction? Having an error on one disk, versus multiple disks in a RAID totally matters.
    - empthought 10y ago
      How do you figure? If the single disk has bad data written to it, it contains bad data. If the RAID has bad data written to it, it contains bad data. Either way, the bad data is bad, because it was undetectably corrupted in memory before being written out.
      - jonathankoren 10y ago
        https://www.youtube.com/watch?v=osS9EaKNSI4 https://www.youtube.com/watch?v=osS9EaKNSI4 Serious question: Do they not teach Hamming codes in undergrad anymore?
        empthought 10y ago
        Do you not understand the problem here? The problem isn't detecting data corruption in storage, which parity checking also helps with. The problem is detecting data corruption in memory. ECC only matters after irrecoverable data loss _in memory_ has occurred. Without ECC the system will continue to operate; with ECC the system will halt without writing data at all, or correct it before storage even comes into play. Writing corrupt data once or writing it a thousand times redundantly with parity data looks exactly the same, if it's allowed to happen. It's a valid write of invalid data. Either way, the data you have on your cheap laptop drive or your six-figure storage array is corrupt. This is why the ancestor commenter is correct about the storage technology being irrelevant to the choice to use or not use ECC. I haven't been in school for many years. They taught Hamming codes, but they also taught garbage-in, garbage-out.
semi-extrinsic 10y ago
> Building machines without ECC is a bad idea. I'd say it depends on the use case. For business storage servers, especially "mission-critical", sure. (Then again, this data was created somewhere; presumably mainly on laptops without ECC. Why is that not a bad idea?) But for a home storage server, ECC is no more critical than buying a UPS, shielded SATA cables, enterprise-grade drives, a high quality power supply, etc. etc. Together these upgrades will easily double the cost of the storage server. Since you'll want an off-site backup solution anyway in case of disaster, why bother paying for an expensive server that only reduces the chance of needing to restore from backup from 0.001% to 0.00001%?
- wyager 10y ago
  Any work we do to improve the reliability of consumer software is limited by the reliability of the underlying machine. If everyone used ECC RAM, it would be marginally more expensive than regular RAM, but it would let us (theoretically) cut the number of real-life crashes by a huge degree. I think if every computer system were like this, it would have non-obvious positive effects on society (like the ability to safely give computers increased responsibility).
  - semi-extrinsic 10y ago
    > it would let us (theoretically) cut the number of real-life crashes by a huge degree Do you have a source for that claim? I know people have done tests with running large molecular simulations on hundreds of GPUs with/without ECC for days on end, and who've come to the conclusions that ECC probably isn't worth it [1]. And this makes sense. Most of a program's memory usage is data. Thus the rare event of a bitflip is most likely to occur in data. Such a bitflip doesn't cause a crash in a consumer system; maybe it changes the color of one pixel in one frame of a video file, or adds some other tiny noise somewhere. [1] http://dl.acm.org/citation.cfm?id=2484774 http://dl.acm.org/citation.cfm?id=2484774
    - wyager 10y ago
      http://research.google.com/pubs/pub35162.html http://research.google.com/pubs/pub35162.html
marcosdumay 10y ago
Fully agree. But notice that a bit-error in a text file in plan storage has some consequences. That same bit-error in a text file in a RAID (or encrypted storage) has very different consequences. Some bad ideas are just worse than others.

by ryao 10y ago

There is nothing special about storage that makes the absence of ECC in a computer any more of a bad idea than it already is. Building machines without ECC is a bad idea.

tracker1 10y ago

When writing redundant (parity data) across multiple storage devices, you're increasing the chance of error without ECC. While there's still a chance/risk with single drive, the effects of such error are not as significant.

empthought 10y ago
The effects are exactly as significant either way: corrupt data is written to disk. Whether the corrupt data is written one time or several times is immaterial.
- jonathankoren 10y ago
  May I introduce you to error correction? Having an error on one disk, versus multiple disks in a RAID totally matters.
  - empthought 10y ago
    How do you figure? If the single disk has bad data written to it, it contains bad data. If the RAID has bad data written to it, it contains bad data. Either way, the bad data is bad, because it was undetectably corrupted in memory before being written out.
    - jonathankoren 10y ago
      https://www.youtube.com/watch?v=osS9EaKNSI4 https://www.youtube.com/watch?v=osS9EaKNSI4 Serious question: Do they not teach Hamming codes in undergrad anymore?
      - empthought 10y ago
        Do you not understand the problem here? The problem isn't detecting data corruption in storage, which parity checking also helps with. The problem is detecting data corruption in memory. ECC only matters after irrecoverable data loss _in memory_ has occurred. Without ECC the system will continue to operate; with ECC the system will halt without writing data at all, or correct it before storage even comes into play. Writing corrupt data once or writing it a thousand times redundantly with parity data looks exactly the same, if it's allowed to happen. It's a valid write of invalid data. Either way, the data you have on your cheap laptop drive or your six-figure storage array is corrupt. This is why the ancestor commenter is correct about the storage technology being irrelevant to the choice to use or not use ECC. I haven't been in school for many years. They taught Hamming codes, but they also taught garbage-in, garbage-out.

semi-extrinsic 10y ago

> Building machines without ECC is a bad idea. I'd say it depends on the use case. For business storage servers, especially "mission-critical", sure. (Then again, this data was created somewhere; presumably mainly on laptops without ECC. Why is that not a bad idea?) But for a home storage server, ECC is no more critical than buying a UPS, shielded SATA cables, enterprise-grade drives, a high quality power supply, etc. etc. Together these upgrades will easily double the cost of the storage server. Since you'll want an off-site backup solution anyway in case of disaster, why bother paying for an expensive server that only reduces the chance of needing to restore from backup from 0.001% to 0.00001%?

wyager 10y ago
Any work we do to improve the reliability of consumer software is limited by the reliability of the underlying machine. If everyone used ECC RAM, it would be marginally more expensive than regular RAM, but it would let us (theoretically) cut the number of real-life crashes by a huge degree. I think if every computer system were like this, it would have non-obvious positive effects on society (like the ability to safely give computers increased responsibility).
- semi-extrinsic 10y ago
  > it would let us (theoretically) cut the number of real-life crashes by a huge degree Do you have a source for that claim? I know people have done tests with running large molecular simulations on hundreds of GPUs with/without ECC for days on end, and who've come to the conclusions that ECC probably isn't worth it [1]. And this makes sense. Most of a program's memory usage is data. Thus the rare event of a bitflip is most likely to occur in data. Such a bitflip doesn't cause a crash in a consumer system; maybe it changes the color of one pixel in one frame of a video file, or adds some other tiny noise somewhere. [1] http://dl.acm.org/citation.cfm?id=2484774 http://dl.acm.org/citation.cfm?id=2484774
  - wyager 10y ago
    http://research.google.com/pubs/pub35162.html http://research.google.com/pubs/pub35162.html

marcosdumay 10y ago

Fully agree. But notice that a bit-error in a text file in plan storage has some consequences. That same bit-error in a text file in a RAID (or encrypted storage) has very different consequences. Some bad ideas are just worse than others.