4 ms·

> https://github.com/hse-project/hse https://github.com/hse-project/hse Their benchmarks show significant gains compared to RocksDB. > https://github.com/spdk

> https://github.com/hse-project/hse https://github.com/hse-project/hse Their benchmarks show significant gains compared to RocksDB. > https://github.com/spdk/rocksdb https://github.com/spdk/rocksdb But what I'd really like to see is a comparison against RocksDB using SPDK > https://dqtibwqq6s6ux.cloudfront.net/download/papers/Hitachi_SPDK_NVMe_oF_Performance_Report.pdf https://dqtibwqq6s6ux.cloudfront.net/download/papers/Hitachi... Based on these results, SPDK performs significantly better than the kernel requiring only 1-2 cores to saturate IOPS on an NVMe SSD (compared to the kernel requiring 16)

wtallis 6y ago
I'd also like to see comparison to Toshiba/Kioxia's fork TRocksDB: https://github.com/KioxiaAmerica/trocksdb https://github.com/KioxiaAmerica/trocksdb
ignoramous 6y ago
SPDK has had its share of detractors here on news.yc [0], especially with io_uring around the block [1]. It'd be interesting to see the improvements to these io-centric applications once they move to io_uring [2], which, in a way, like RocksDB, is sponsored by Facebook [3]. [0] https://news.ycombinator.com/item?id=10511960 https://news.ycombinator.com/item?id=10511960 [1] https://news.ycombinator.com/item?id=22266503 https://news.ycombinator.com/item?id=22266503 [2] https://news.ycombinator.com/item?id=19843464 https://news.ycombinator.com/item?id=19843464 [3] https://lkml.org/lkml/2014/1/24/252 https://lkml.org/lkml/2014/1/24/252
- Rafuino 6y ago
  SPDK has made some progress since [0] was posted in 2015 (see https://spdk.io/ https://spdk.io/ if you're curious). Still, I am excited about io_uring and what it brings to the table.
- benlwalker 6y ago
  io_uring is a fantastic development for the kernel, and I really can't praise it enough. However, there's still lots of reasons to use SPDK. Performance is still significantly better[0], and you can directly access all the of the NVMe features on the device without going through any abstraction layers. [0] https://spdk.io/news/2019/05/06/nvme/ https://spdk.io/news/2019/05/06/nvme/
  - zerd 6y ago
    Woah, just realizing that 10.39M 4k IOPS is 42GB/s. Doing 40GB/s of sequential IO was difficult not that many years ago, let alone random IO. That's faster than my memory bandwidth. https://www.microway.com/knowledge-center-articles/performance-characteristics-of-common-transports-buses/ https://www.microway.com/knowledge-center-articles/performan...
- teleforce 6y ago
  For those who is wondering about what is io_uring and its significance I'd highly recommend this blog article posted at HN few days ago[1]. The potential of using io_uring with eBPF in Linux will makes any HPC enthusiast drooling :-) https://news.ycombinator.com/item?id=22974728 https://news.ycombinator.com/item?id=22974728
AtlasBarfed 6y ago
Is there an api compatibility for this thing so it can be switched easily out for things like Rocksandra?
benlwalker 6y ago
SPDK's RocksDB integration really hasn't gotten a lot of love. There's really two main challenges we hit and then never revisited. First, the IO threads in RocksDB are a thread pool that assume they perform blocking operations. That doesn't jive with SPDK's async model (nor io_uring's). We're having to message pass to an async thread and block on semaphores in the IO threads. Second, RocksDB was heavily reliant on the kernel page cache to make it fast. Both of these things could have changed since we last worked on the integration. I haven't kept up with RocksDB development recently. source: am SPDK maintainer
- continuations 6y ago
  The SPDK RocksDB repo (https://github.com/spdk/rocksdb https://github.com/spdk/rocksdb) seems to have been abandoned (last commit 2017). Is there a maintained version of SPDK RocksDB? Or SPDK any DB?

wtallis 6y ago

I'd also like to see comparison to Toshiba/Kioxia's fork TRocksDB: https://github.com/KioxiaAmerica/trocksdb https://github.com/KioxiaAmerica/trocksdb

ignoramous 6y ago

SPDK has had its share of detractors here on news.yc [0], especially with io_uring around the block [1]. It'd be interesting to see the improvements to these io-centric applications once they move to io_uring [2], which, in a way, like RocksDB, is sponsored by Facebook [3]. [0] https://news.ycombinator.com/item?id=10511960 https://news.ycombinator.com/item?id=10511960 [1] https://news.ycombinator.com/item?id=22266503 https://news.ycombinator.com/item?id=22266503 [2] https://news.ycombinator.com/item?id=19843464 https://news.ycombinator.com/item?id=19843464 [3] https://lkml.org/lkml/2014/1/24/252 https://lkml.org/lkml/2014/1/24/252

Rafuino 6y ago
SPDK has made some progress since [0] was posted in 2015 (see https://spdk.io/ https://spdk.io/ if you're curious). Still, I am excited about io_uring and what it brings to the table.
benlwalker 6y ago
io_uring is a fantastic development for the kernel, and I really can't praise it enough. However, there's still lots of reasons to use SPDK. Performance is still significantly better[0], and you can directly access all the of the NVMe features on the device without going through any abstraction layers. [0] https://spdk.io/news/2019/05/06/nvme/ https://spdk.io/news/2019/05/06/nvme/
- zerd 6y ago
  Woah, just realizing that 10.39M 4k IOPS is 42GB/s. Doing 40GB/s of sequential IO was difficult not that many years ago, let alone random IO. That's faster than my memory bandwidth. https://www.microway.com/knowledge-center-articles/performance-characteristics-of-common-transports-buses/ https://www.microway.com/knowledge-center-articles/performan...
teleforce 6y ago
For those who is wondering about what is io_uring and its significance I'd highly recommend this blog article posted at HN few days ago[1]. The potential of using io_uring with eBPF in Linux will makes any HPC enthusiast drooling :-) https://news.ycombinator.com/item?id=22974728 https://news.ycombinator.com/item?id=22974728

AtlasBarfed 6y ago

Is there an api compatibility for this thing so it can be switched easily out for things like Rocksandra?

benlwalker 6y ago

SPDK's RocksDB integration really hasn't gotten a lot of love. There's really two main challenges we hit and then never revisited. First, the IO threads in RocksDB are a thread pool that assume they perform blocking operations. That doesn't jive with SPDK's async model (nor io_uring's). We're having to message pass to an async thread and block on semaphores in the IO threads. Second, RocksDB was heavily reliant on the kernel page cache to make it fast. Both of these things could have changed since we last worked on the integration. I haven't kept up with RocksDB development recently. source: am SPDK maintainer

continuations 6y ago
The SPDK RocksDB repo (https://github.com/spdk/rocksdb https://github.com/spdk/rocksdb) seems to have been abandoned (last commit 2017). Is there a maintained version of SPDK RocksDB? Or SPDK any DB?