3 ms·
A ways back, I wrote a sort of database that was memory-mapped-file backed (a mistake, but I didn’t know that at the time), and I would have paid top dollar for
by amluto 7mo ago
A ways back, I wrote a sort of database that was memory-mapped-file backed (a mistake, but I didn’t know that at the time), and I would have paid top dollar for even a few GB of NVDIMMs that could be put in an ordinary server and could be somewhat straightforwardly mounted as a DAX filesystem. I even tried to do some of the kernel work. But the hardware and firmware was such a mess that it was basically a lost cause. And none of the tech ever seemed to turn into an actual purchasable product. I’m a bit suspicious that Intel never found product-market fit in part because they never had a credible product on the NVDIMM side.
Somewhere I still have some actual battery-backed DIMMs (DRAM plus FPGA interposer plus awkward little supercapacitor bundle) in a drawer. They were not made by Intel, but Intel was clearly using them as a stepping stone toward the broader NVDIMM ecosystem. They worked on exactly one SuperMicro board, kind of, and not at all if you booted using UEFI. Rebooting without doing the magic handshake over SMBUS [0] first took something like 15 minutes, which was not good for those nines of availability.
[0] You can find my SMBUS host driver for exactly this purpose on the LKML archives. It was never merged, in part, because no one could ever get all the teams involved in the Xeon memory controller to reach any sort of agreement as to who owned the bus or how the OS was supposed to communicate without, say, defeating platform thermal management or causing the refresh interval to get out of sync with the DIMM temperature, thus causing corruption.
I’m suspicious that everything involved in Optane development was like this.
- simulator5g 7mo agoIf you're thinking about reliability in terms of 9s you should probably not be depending on a single machine. Reboots could take hours and be fine if your architecture is set up for reliability.
- amluto 7mo agoThings go down and need to come back up. Take a look at almost any major cloud incident in the last 15 years — some combination of goofs results in a bunch of services going down. Then it takes many minutes or hours to come back up. Sure, one can (and should) try to design for fewer failures that result in waiting for things to come up, but one should also design for faster bringup. For the NVDIMM in question, the whole point is for durability after a reboot or power loss. A fifteen minute cycle in which it would write its contents out to nonvolatile storage and then read it back, during which the firmware is too inept to notice that all the data is already loaded is a bug that should have been fixed. (In fairness, this was a pre-production model.) Even ignoring the reboot time, SMBUS was needed to properly identify the device and to access its health-check data. As for my database, it’s not intended for active-active HA setups — it’s intended for use on a small number of high quality machines. It has had zero meaningful data loss incidents in its entire time in production, and it’s extremely fast. But it has some other properties I don’t like and that would cause me to do it differently if I were to start over.