4 ms·
I don't remember exactly why I have any of them saved, but these are some experimental data stores that seems to be fitting what you're looking for somewhat: -
by diggan 3y ago
I don't remember exactly why I have any of them saved, but these are some experimental data stores that seems to be fitting what you're looking for somewhat:
- https://github.com/DataManagementLab/ScaleStore https://github.com/DataManagementLab/ScaleStore - "A Fast and Cost-Efficient Storage Engine using DRAM, NVMe, and RDMA"
- https://github.com/unum-cloud/udisk https://github.com/unum-cloud/udisk (https://github.com/unum-cloud/ustore https://github.com/unum-cloud/ustore) - "The fastest ACID-transactional persisted Key-Value store designed for NVMe block-devices with GPU-acceleration and SPDK to bypass the Linux kernel."
- https://github.com/capsuleman/ssd-nvme-database https://github.com/capsuleman/ssd-nvme-database - "Columnar database on SSD NVMe"
- ashvardanian 3y agoHey, thanks for the mention! UDisk, however, hasn't been open-sourced yet. Still considering it :)
- geek_at 3y agoyou could also configure Redis to transact everything to disk and choose nvme as the target
- jitl 3y agoThat would save via file system, not bypass the kernel to access the NVMe drive directly from user space. NVMe drives themself have a bunch of features that make them amenable to K/V storage directly. Good overview: https://www.mydistributed.systems/2020/07/towards-building-high-performance-scale.html?m=1 https://www.mydistributed.systems/2020/07/towards-building-h...
- PaulHoule 3y agoSee https://www.snia.org/educational-library/key-value-standardized-2020 https://www.snia.org/educational-library/key-value-standardi... for some description of the special command set to get an nvme drive to natively work as a key-value store. Also https://www.snia.org/sites/default/files/ESF/Key-Value-Storage-Standard-Final.pdf https://www.snia.org/sites/default/files/ESF/Key-Value-Stora...
- mycall 3y agoHow is that implemented? Btree, hashtable?
- anticensor 3y agoImplementation-defined. The API resembles a map access, though.
- aftbit 3y agoInteresting, never heard of this before! Do you have any other resources to share? How can I play with this today?
- laurencerowe 3y agoHow do you tell which NVMe drive models support the KV API? Is this something that you can experiment with on a consumer drive or do you need specific enterprise ssd models? Samsung's uNVMe evaluation guide (from 2019) device support section just states: Guide Version: uNVMe2.0 SDK Evaluation Guide ver 1.2 Supported Product(s): NVMe SSD (Block/KV) Interface(s): NVMe 1.2 https://github.com/OpenMPDK/uNVMe/blob/master/doc/uNVMe2.0_SDK_Evaluation_Guide_v1.2.pdf https://github.com/OpenMPDK/uNVMe/blob/master/doc/uNVMe2.0_S... I can't find detailed spec sheets detailing which NVMe command sets are supported even for their enterprise drives.
- laurencerowe 3y agoI reached out to Samsung support to ask. After being sent from one department to another and receiving some very clearly incorrect advice from their sales support they eventually sent me to an online form for the memory department. Am still waiting for a response a week later.
- jamesblonde 3y agoRonDB is open-source and supports on-disk data on NVMe disks. http://mikaelronstrom.blogspot.com/2022/04/variable-sized-disk-rows-in-rondb.html http://mikaelronstrom.blogspot.com/2022/04/variable-sized-di...