3 ms·

I don’t think I’ve ever had ECC, and have never had any issues. What kind of problems would you expect to see?

by Aeolun 2y ago

averageRoyalty 2y ago
Non shielded RAM is subject to bit flipping. Non-ECC always carries this risk in general computing, but the problem is compounded when you run a filesystem like ZFS which uses memory as a significant storage element for write cache. If it would hugely impact your life if a bit were flipped from a 0 to 1 in your stored data - say you make videos or store your bitcoin wallet key on your NAS - you are running a risk not using ECC. You may not have had issues or ever have issues with non-ECC. Your car may never be stolen if you leave the keys in either, but it's not a good risk proposition for most people.
toast0 2y ago
Well part of it is you likely won't see the problem until a long time after it happens. But on servers with ECC and reporting I saw several different patterns: a) 99%+ (or something) of the servers had zero reported errors for their lifetime. Memory usually works, no problems experienced. b) Some of the servers reported one error, one time, and then all was well for the rest of their life. If this happens without ECC, and you're lucky, it's in some memory that doesn't really matter and it's no big deal. Or maybe it crashes your kernel because it's in the right spot (flip a bit in a pointer/return address and it's easy to crash). Or maybe it managed to flip a bit in a pending write and your file is stored wrong and the zfs checksum is calculated on the wrong data. If you're really unlucky, you could probably write bad metadata and have a real hard time mounting the filesystem later? c) some servers reported the same address with a correctable error once a day; probably one bit stuck, any time data transits that address, that bit is still stuck, and that will likely cause trouble. If it's used for kernel memory, you'll probably end up crashing sooner or later. d) some servers had a lot more ram errors; sometimes a slow ramp to a hundred a day, once or twice a rapid ramp to so many that the system spent most of its time handling machine check exceptions, but did manage to stay online but developed a queue it could never process. Once you're at these levels, you'll probably get crashes, but you might write some bad data first. Ram testing helps on systems without ECC, but you won't really know if/when the ram has gone from working 100% to working almost 100%. I have a desktop that was running fine for a while, tests fine, but crashes in ways that have to be bad ram, and removing one stick seems to have fixed it.

I don’t think I’ve ever had ECC, and have never had any issues. What kind of problems would you expect to see?

by Aeolun 2y ago

I don’t think I’ve ever had ECC, and have never had any issues. What kind of problems would you expect to see?

Non shielded RAM is subject to bit flipping. Non-ECC always carries this risk in general computing, but the problem is compounded when you run a filesystem like ZFS which uses memory as a significant storage element for write cache. If it would hugely impact your life if a bit were flipped from a 0 to 1 in your stored data - say you make videos or store your bitcoin wallet key on your NAS - you are running a risk not using ECC. You may not have had issues or ever have issues with non-ECC. Your car may never be stolen if you leave the keys in either, but it's not a good risk proposition for most people.

toast0 2y ago

Well part of it is you likely won't see the problem until a long time after it happens. But on servers with ECC and reporting I saw several different patterns: a) 99%+ (or something) of the servers had zero reported errors for their lifetime. Memory usually works, no problems experienced. b) Some of the servers reported one error, one time, and then all was well for the rest of their life. If this happens without ECC, and you're lucky, it's in some memory that doesn't really matter and it's no big deal. Or maybe it crashes your kernel because it's in the right spot (flip a bit in a pointer/return address and it's easy to crash). Or maybe it managed to flip a bit in a pending write and your file is stored wrong and the zfs checksum is calculated on the wrong data. If you're really unlucky, you could probably write bad metadata and have a real hard time mounting the filesystem later? c) some servers reported the same address with a correctable error once a day; probably one bit stuck, any time data transits that address, that bit is still stuck, and that will likely cause trouble. If it's used for kernel memory, you'll probably end up crashing sooner or later. d) some servers had a lot more ram errors; sometimes a slow ramp to a hundred a day, once or twice a rapid ramp to so many that the system spent most of its time handling machine check exceptions, but did manage to stay online but developed a queue it could never process. Once you're at these levels, you'll probably get crashes, but you might write some bad data first. Ram testing helps on systems without ECC, but you won't really know if/when the ram has gone from working 100% to working almost 100%. I have a desktop that was running fine for a while, tests fine, but crashes in ways that have to be bad ram, and removing one stick seems to have fixed it.