4 ms·

NVMe devices can be split into regions with a different block size per region. If you look at the spec and are interested in building a storage system on top of

• jlokier 5y ago
  I tried to split an NVMe into different blocks sizes per region recently. It was to get the small performance improvement (~2%) of 4096 byte sectors, while needing to boot on a motherboard that needs 512 byte sectors (I didn't want to deal with switching a remote server to UEFI and figure out every issue remotely; it's a shame there isn't some trivial BIOS extension to support 4096 byte sectors). So I decided to format a boot region and main region. What I found with the particular models I had was: - Multiple regions were fine, and different block sizes (512 or 4096 bytes) were fine too, but they wouln't accept different block sizes in different regions at the same time. Format commands failed when you tried that. - The atomicity size and alignment were identical regardless of block size. Durable (power fail safe) atomicity was reported as 4096 byte aligned blocks. Write command overlapping atomicity was reported as 512kiB aligned blocks. I'm not sure why you may think I think NVMe resembles a HDD regarding atomicity. HDDs don't claim to do atomic replacement of sector contents, and neither do non-NVMe SSDs. However, NVMe specifies the atomicity, and the device reports atomicity parameters. Are you saying the NVMe devices lie about their atomicity as well? If so, that's a real shame, after going to the effort of reporting atomicity parameters in the first place. Or are you inferring it from experience with HDDs and non-NVMe SSDs? With SSDs (at least low-quality non-NVMe ones) I'm not convinced you can make use of "you don't break what you don't touch". Blocks are remapped all over the place, so you don't know which banks you are touching, so if there are faults with the SSD you don't know which random blocks may become corrupt as a side effect of the blocks you are writing to, so you can't work around it. In other words, such devices are not usable for anything if you need power fail safety. Atomicity is the least of your worries. The most extreme awfulness I've heard of is USB flash reporting fake capacity. So when you write your important files to it, it's guaranteed to corrupt them. https://www.sqlite.org/howtocorrupt.html#_fake_capacity_usb_sticks https://www.sqlite.org/howtocorrupt.html#_fake_capacity_usb_... However, to the extent you decide to depend on a device not corrupting random unrelated data on power failure, it's hard to see why you should not include the NVMe-reported block atomicity in that assumption and modify your storage algorithm to take advantage of it. Of course any device can fail catastrophically to meet its own basic specifications, such as your bit 2 in every byte example, and in general random corruption due to design or manufacturing issues. It doesn't make sense to design around those kinds of "unknown unknowns" at the B-tree and logging format design level though. Those need to be handled at a higher level, where we are willing to write off a database and perhaps switch to a backup. Edit: ps. Thanks for the Usenix paper link. It's a good paper, and I'd hoped to see someone do those measurements. I hope to see a similar paper about NVMe someday.
  • toolslive 5y ago
    | Are you saying the NVMe devices lie about their atomicity as well. I've seen 'em do it, man.