3 ms·

Seemingly faster and more compact than lmdb-go, bbolt, Badger, and goleveldb at <2K LOC. goos: linux goarch: arm64 pkg: github.com/voidDB/voidDB/test B

Seemingly faster and more compact than lmdb-go, bbolt, Badger, and goleveldb at <2K LOC. goos: linux goarch: arm64 pkg: github.com/voidDB/voidDB/test BenchmarkVoidPut-2 131072 14933 ns/op BenchmarkVoidGet-2 131072 1060 ns/op BenchmarkVoidGetNext-2 131072 245.8 ns/op BenchmarkLMDBPut-2 131072 22414 ns/op BenchmarkLMDBGet-2 131072 1826 ns/op BenchmarkLMDBGetNext-2 131072 602.2 ns/op BenchmarkBoltPut-2 131072 66984 ns/op BenchmarkBoltGet-2 131072 2552 ns/op BenchmarkBoltGetNext-2 131072 254.6 ns/op BenchmarkLevelPut-2 131072 44182 ns/op BenchmarkLevelGet-2 131072 30949 ns/op BenchmarkLevelGetNext-2 131072 3441 ns/op BenchmarkBadgerPut-2 131072 15182 ns/op BenchmarkBadgerGet-2 131072 33114 ns/op BenchmarkBadgerGetNext-2 131072 12895 ns/op BenchmarkNothing-2 131072 0.3239 ns/op Check out the docs at https://pkg.go.dev/github.com/voidDB/voidDB https://pkg.go.dev/github.com/voidDB/voidDB

KAdot 2y ago
The benchmark doesn't accurately represent the real-world database performance because the dataset is too small (roughly half a gigabyte based on [1]?), meaning it fits into the page cache bypassing disk I/O. [1]: https://github.com/voidDB/voidDB/blob/master/test/bench_test.go https://github.com/voidDB/voidDB/blob/master/test/bench_test...
- chronode 2y ago
  This is a cool effort but their claims are wildly misleading as their benchmarks aren't fair across the board: For example: - BenchmarkVoidPut runs a single O(n) sized transaction - BenchmarkLMDBPut runs a single O(n) sized transaction - BenchmarkBadgetPut runs O(n) O(n) sized transactions (!!!) I made a local change so that all Put benchmarks ran O(n) O(1) sized transactions and the results were quite different: Void was the slowest, followed by LMDB, Bold, LevelDB, then Badger. I'd also wager the LMDB author would also (lovingly!) tell us we're holding it wrong
  - arandomusername 2y ago
    LMDB is the only one here in C, so the interop is probably what makes LMDB so slow in here
    - apatheticonion 2y ago
      Curious, what overhead is there calling C code from other languages?
      - atombender 2y ago
        Go doesn't use the C calling convention, but has its own growable stack system and goroutine scheduler that maps to goroutines to threads. So a goroutine can't just call a C function directly. In order to interface with C code safely, Go's runtime has to jump to the system stack and do some additional setup, make the call, and then switch back. (Adding to that, if the call takes too long, this prevents other goroutines on the same OS thread from running, so the scheduler must jump in and move those goroutines to a different thread.) All of this is expensive, though we are talking about nanoseconds, not milliseconds. Performance is mostly a problem when doing lots of very quick calls (e.g. you're writing a game engine interacting with something like OpenGL) or lots of slow calls (causing scheduler trashing).
        apatheticonion 2y ago
        Thanks! Is this also the case even Rust consumes a C library?
        atombender 2y ago
        No, my understanding is that Rust uses normal stacks, and it uses a classic threading model, so aside from async, calling C doesn't need to any runtime stuff.
        steveklabnik 2y ago

KAdot 2y ago

The benchmark doesn't accurately represent the real-world database performance because the dataset is too small (roughly half a gigabyte based on [1]?), meaning it fits into the page cache bypassing disk I/O. [1]: https://github.com/voidDB/voidDB/blob/master/test/bench_test.go https://github.com/voidDB/voidDB/blob/master/test/bench_test...

chronode 2y ago
This is a cool effort but their claims are wildly misleading as their benchmarks aren't fair across the board: For example: - BenchmarkVoidPut runs a single O(n) sized transaction - BenchmarkLMDBPut runs a single O(n) sized transaction - BenchmarkBadgetPut runs O(n) O(n) sized transactions (!!!) I made a local change so that all Put benchmarks ran O(n) O(1) sized transactions and the results were quite different: Void was the slowest, followed by LMDB, Bold, LevelDB, then Badger. I'd also wager the LMDB author would also (lovingly!) tell us we're holding it wrong
- arandomusername 2y ago
  LMDB is the only one here in C, so the interop is probably what makes LMDB so slow in here
  - apatheticonion 2y ago
    Curious, what overhead is there calling C code from other languages?
    - atombender 2y ago
      Go doesn't use the C calling convention, but has its own growable stack system and goroutine scheduler that maps to goroutines to threads. So a goroutine can't just call a C function directly. In order to interface with C code safely, Go's runtime has to jump to the system stack and do some additional setup, make the call, and then switch back. (Adding to that, if the call takes too long, this prevents other goroutines on the same OS thread from running, so the scheduler must jump in and move those goroutines to a different thread.) All of this is expensive, though we are talking about nanoseconds, not milliseconds. Performance is mostly a problem when doing lots of very quick calls (e.g. you're writing a game engine interacting with something like OpenGL) or lots of slow calls (causing scheduler trashing).
      - apatheticonion 2y ago
        Thanks! Is this also the case even Rust consumes a C library?
        atombender 2y ago
        No, my understanding is that Rust uses normal stacks, and it uses a classic threading model, so aside from async, calling C doesn't need to any runtime stuff.
        steveklabnik 2y ago