3 ms·

There's actually an NVMe command set which allows you to use the FTL directly as a K/V store. (This is limited to 16-byte keys [1] however, so it is not that us

There's actually an NVMe command set which allows you to use the FTL directly as a K/V store. (This is limited to 16-byte keys [1] however, so it is not that useful and probably not implemented anywhere, my guess is Samsung looked at this for some hyperscaler, whipped up a prototype in their customer-specific firmware and the benefits were lesser than expected so it's dead now) [1] These slides claim up to 32 bytes, which would be a practically useful length: https://www.snia.org/sites/default/files/ESF/Key-Value-Storage-Standard-Final.pdf https://www.snia.org/sites/default/files/ESF/Key-Value-Stora... but the current revision of the standard only permits two 64-bit words as the key ("The maximum KV key size is 16 bytes"): https://nvmexpress.org/wp-content/uploads/NVM-Express-Key-Value-Command-Set-Specification-1.0c-2022.10.03-Ratified-1.pdf https://nvmexpress.org/wp-content/uploads/NVM-Express-Key-Va...

londons_explore 3y ago
Presumably there is some way to use the hash of the actual key as the key, and then store both key and value as data? 16 bytes is long enough that collisions will be super rare, and while you obviously need to write code to support that case, it should have no performance impact.
- deleted 3y ago
  [deleted]
- formerly_proven 3y ago
  A 32-byte key would allow using NVMe KV directly for content-addressed storage; many of those systems use 256-bit / 32-byte cryptographic hashes as keys. Notable exception would be git with 20-byte keys.
  - londons_explore 3y ago
    You can still do this... Just use the first 16 bytes as the key, and the 2nd 16 bytes as the start of the data.
londons_explore 3y ago
I could imagine that if this mode isn't widely used, drive manufacturers haven't given much thought to performance, and it therefore might suck.
londons_explore 3y ago
I think some devices built the block storage on top of the key-value store. Ie. when you write "hello..." (4k bytes) to address 123, it actually saves key: 123, value "hello...". If so, that is probably the reason for a 16 byte key - there is just no way anybody needs a key bigger than 16 bytes for an address anytime soon.

londons_explore 3y ago

Presumably there is some way to use the hash of the actual key as the key, and then store both key and value as data? 16 bytes is long enough that collisions will be super rare, and while you obviously need to write code to support that case, it should have no performance impact.

deleted 3y ago
[deleted]
formerly_proven 3y ago
A 32-byte key would allow using NVMe KV directly for content-addressed storage; many of those systems use 256-bit / 32-byte cryptographic hashes as keys. Notable exception would be git with 20-byte keys.
- londons_explore 3y ago
  You can still do this... Just use the first 16 bytes as the key, and the 2nd 16 bytes as the start of the data.

londons_explore 3y ago

I could imagine that if this mode isn't widely used, drive manufacturers haven't given much thought to performance, and it therefore might suck.

londons_explore 3y ago

I think some devices built the block storage on top of the key-value store. Ie. when you write "hello..." (4k bytes) to address 123, it actually saves key: 123, value "hello...". If so, that is probably the reason for a 16 byte key - there is just no way anybody needs a key bigger than 16 bytes for an address anytime soon.