6 ms·
I have been working on a hobby database that uses a key-value store as the backend. Currently, it is structured as a binary tree such that an append-only log is
by bob1029 7y ago
I have been working on a hobby database that uses a key-value store as the backend. Currently, it is structured as a binary tree such that an append-only log is possible (ideal for handling updates on a traditional SSD). This is a very complex and time-consuming part of the implementation.
With this KV drive, I could potentially 'hardware accelerate' my database backend and all of the complexity of log-structured merge trees falls away. It would just be a handful of calls into a KV SSD library that handles the magic of getting a KV pair to disk. All I would need to worry about are the higher-order primitives I can build on top of a KV store. Using hashing schemes and clever metadata structures, you can encode virtually unlimited amounts of information into a 256 bit key. One simple flat store of KV items can contain all of your indexes, objects, scripts, jobs, settings, etc. Additionally, using hashed metadata keys lends itself well to sharing the store across unlimited drives and nodes. 256 bits comprises an unimaginably large key space, and SHA512 is actually even faster to execute on most x86 64-bit hardware if you can eat the 2x overhead on keys (which can be safely assumed to be inexhaustible through the heat death of the universe at this point).