3 ms·

The triple reading/writing to memory along with the comparing would probably be a significant performance hit. You could just use a bit of extra memory for pari

The triple reading/writing to memory along with the comparing would probably be a significant performance hit. You could just use a bit of extra memory for parity bits etc. instead

magicalhippo 2y ago
Sure it would be a significant hit. But only for relatively few pages, and I imagine most of them would either stay cached or are very cold. Of course you could trade implementation complexity for speed as always. My main point was to have effective ECC without any additional support from motherboard and memory modules.
- menaerus 2y ago
  The important detail to make your theory plausible, and without a huge performance impact and/or jittering, is how the kernel memory management system is going to decide what exactly is those "relatively few pages" on a server that runs a plentiful of processes, many of them including critical/heavy ones such as databases? One such example comes to my mind - transparent huge pages. And it didn't turn out to be quite successful. I imagine soft-ECC would be more plausible if it could be applied/enabled: 1. Per-process (e.g. whole application) 2. Or fine-grained per-allocation within a process/application I think both could be implemented through a (system) memory allocator by taking the advantage of page alignment LSBs and/or (non) addressable bits of the virtual memory. Those spare bits in memory can be used to store the encoding of ECC algorithm (Hamming, Reed-Solomon, or something more primitive but less robust). And then, depending on the ECC algo, one could substantially minimize the performance impact of encode/decode by using SIMD.
  - magicalhippo 2y ago
    I imagined executable pages and pages explicitly marked by the application/kernel via VirtualProtect[1] or similar. Of course an application that marks GB of data this way could play havoc with other processes, so maybe some OS limits on how many non-executable pages a process can mark may be needed. And there certainly might be some further dragons I'm not considering. [1]: https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-virtualprotect https://learn.microsoft.com/en-us/windows/win32/api/memoryap...

The triple reading/writing to memory along with the comparing would probably be a significant performance hit. You could just use a bit of extra memory for parity bits etc. instead

magicalhippo 2y ago

Sure it would be a significant hit. But only for relatively few pages, and I imagine most of them would either stay cached or are very cold. Of course you could trade implementation complexity for speed as always. My main point was to have effective ECC without any additional support from motherboard and memory modules.

menaerus 2y ago
The important detail to make your theory plausible, and without a huge performance impact and/or jittering, is how the kernel memory management system is going to decide what exactly is those "relatively few pages" on a server that runs a plentiful of processes, many of them including critical/heavy ones such as databases? One such example comes to my mind - transparent huge pages. And it didn't turn out to be quite successful. I imagine soft-ECC would be more plausible if it could be applied/enabled: 1. Per-process (e.g. whole application) 2. Or fine-grained per-allocation within a process/application I think both could be implemented through a (system) memory allocator by taking the advantage of page alignment LSBs and/or (non) addressable bits of the virtual memory. Those spare bits in memory can be used to store the encoding of ECC algorithm (Hamming, Reed-Solomon, or something more primitive but less robust). And then, depending on the ECC algo, one could substantially minimize the performance impact of encode/decode by using SIMD.
- magicalhippo 2y ago
  I imagined executable pages and pages explicitly marked by the application/kernel via VirtualProtect[1] or similar. Of course an application that marks GB of data this way could play havoc with other processes, so maybe some OS limits on how many non-executable pages a process can mark may be needed. And there certainly might be some further dragons I'm not considering. [1]: https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-virtualprotect https://learn.microsoft.com/en-us/windows/win32/api/memoryap...