5 ms·

You were already losing credibility due to your inattention to detail. I never mentioned performance anywhere in this thread. It's stupid to even talk about it

You were already losing credibility due to your inattention to detail. I never mentioned performance anywhere in this thread. It's stupid to even talk about it since a hash doesn't do ordered lookups. But now that you're raising the topic: https://github.com/hyc/leveldb/tree/benches/doc/bench https://github.com/hyc/leveldb/tree/benches/doc/bench "Spanks the living crap" - must be some heretofore unknown definition of "spanks". ./db_bench_mdbm MDBM: version 4.11.1 Date: Mon Dec 15 04:39:20 2014 CPU: 4 * Intel(R) Core(TM)2 Extreme CPU Q9300 @ 2.53GHz CPUCache: 6144 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) ------------------------------------------------ fillrandsync : 40.627 micros/op 24614 ops/sec; 2.7 MB/s (1000 ops) 65604 /tmp/leveldbtest-1000 fillrandom : 16.356 micros/op 61137 ops/sec; 6.8 MB/s 122056 /tmp/leveldbtest-1000 fillrandbatch : 5.499 micros/op 181850 ops/sec; 20.1 MB/s 121936 /tmp/leveldbtest-1000 fillseqsync : 40.163 micros/op 24898 ops/sec; 2.8 MB/s (1000 ops) 65604 /tmp/leveldbtest-1000 fillseq : 16.424 micros/op 60886 ops/sec; 6.7 MB/s 175724 /tmp/leveldbtest-1000 fillseqbatch : 5.648 micros/op 177041 ops/sec; 19.6 MB/s 175724 /tmp/leveldbtest-1000 overwrite : 16.290 micros/op 61385 ops/sec; 6.8 MB/s 175724 /tmp/leveldbtest-1000 readrandom : 0.598 micros/op 1672444 ops/sec; (1000000 of 1000000 found) readseq : 0.096 micros/op 10370216 ops/sec; 1147.2 MB/s ./db_bench_mdb LMDB: version LMDB 0.9.14: (September 20, 2014) Date: Mon Dec 15 04:41:37 2014 CPU: 4 * Intel(R) Core(TM)2 Extreme CPU Q9300 @ 2.53GHz CPUCache: 6144 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) ------------------------------------------------ fillrandsync : 12.818 micros/op 78015 ops/sec; 8.6 MB/s (1000 ops) 224 /tmp/leveldbtest-1000/dbbench_mdb-1 224 /tmp/leveldbtest-1000 fillrandom : 4.275 micros/op 233923 ops/sec; 25.9 MB/s 116548 /tmp/leveldbtest-1000/dbbench_mdb-2 116548 /tmp/leveldbtest-1000 fillrandbatch : 3.490 micros/op 286502 ops/sec; 31.7 MB/s 126384 /tmp/leveldbtest-1000/dbbench_mdb-3 126384 /tmp/leveldbtest-1000 fillseqsync : 14.972 micros/op 66791 ops/sec; 7.4 MB/s (1000 ops) 172 /tmp/leveldbtest-1000/dbbench_mdb-4 172 /tmp/leveldbtest-1000 fillseq : 2.231 micros/op 448145 ops/sec; 49.6 MB/s 125872 /tmp/leveldbtest-1000/dbbench_mdb-5 125872 /tmp/leveldbtest-1000 fillseqbatch : 0.425 micros/op 2355457 ops/sec; 260.6 MB/s 125872 /tmp/leveldbtest-1000/dbbench_mdb-6 125872 /tmp/leveldbtest-1000 overwrite : 4.881 micros/op 204857 ops/sec; 22.7 MB/s 125872 /tmp/leveldbtest-1000/dbbench_mdb-6 125872 /tmp/leveldbtest-1000 readrandom : 1.166 micros/op 857624 ops/sec; (1000000 of 1000000 found) readseq : 0.059 micros/op 17092556 ops/sec; 1890.9 MB/s readreverse : 0.042 micros/op 23814626 ops/sec; 2634.5 MB/s Feel free to submit a patch if I got anything wrong in that driver; it was a pretty hasty patch. This is running on tmpfs, so no I/O involved. MDBM is faster on random read, which is what you'd expect since it's a hash and doesn't have to navigate down a tree to locate a record. Aside from that, it's pretty pedestrian.

cbsmith 12y ago
> This is running on tmpfs, so no I/O involved. Hmm... I think that might alter the results actually. > which is what you'd expect since it's a hash and doesn't have to navigate down a tree to locate a record. mdbm uses a hash to select a page, but it actually does store the keys within a page in order. It's kind of a funky mix.
luckydude 12y ago
From the OP's posting: Test MDBM LevelDB KyotoCabinet BerkeleyDB Write Time 1.1 μs 4.5 μs 5.1 μs 14.0 μs Read Time 0.45 μs 5.3 μs 4.9 μs 8.4 μs Sequential Read 0.05 μs 0.53 μs 1.71 μs 39.1 μs Sync Write 2625 μs 34944 μs 177169 μs 13001 μs As to losing credibility, I'm semi retired, I stopped trying to impress people a decade ago. If i were you I'd be more worried about your own image, bad mouthing other people's tech when you demonstrate you don't understand it hasn't made you like good to at least a few people here. It wouldn't be that hard to say "MDBM is great when used as an index into a DB, it works just fine for that. But you are going to have to rebuild the index after a power failure unless you take care to flush the data. It's somewhat unfair that the OP compared against my database because mine is slower because it handles crashes." Instead you come out with "MDBM is complete crap". Well, no, it's not. In the domain where it is useful it is actually quite useful, it's 10x faster for lookups than your DB. So the trade off is speed vs surviving reboots. For lots of people, speed is much more important. Machines don't crash every ten minutes. In fact, it's pretty common to see uptimes in 100's of days. Lets say that 100 days is average and lets say that it takes a full day to rebuild the MDBM. So MDBM is delivering 99/100 days of useful work. You deliver 100/100 days. Oh, wait, except that your useful work is running 10x slower if the DB is being used as an index. So you delivered 10/100 days. See why some people may prefer to use MDBM when it is put like that? Performance is a feature.
- cbsmith 12y ago
  I think the comparison was totally appropriate because the article specifically highlighted mdbm's design limitations. The whole point was that given the compromises mdbm makes, it is possible to yield significant performance gains, so if those design limitations fit your context (and it cited 4 specific examples where Yahoo uses them), you ought to consider it.

cbsmith 12y ago

> This is running on tmpfs, so no I/O involved. Hmm... I think that might alter the results actually. > which is what you'd expect since it's a hash and doesn't have to navigate down a tree to locate a record. mdbm uses a hash to select a page, but it actually does store the keys within a page in order. It's kind of a funky mix.

luckydude 12y ago

From the OP's posting: Test MDBM LevelDB KyotoCabinet BerkeleyDB Write Time 1.1 μs 4.5 μs 5.1 μs 14.0 μs Read Time 0.45 μs 5.3 μs 4.9 μs 8.4 μs Sequential Read 0.05 μs 0.53 μs 1.71 μs 39.1 μs Sync Write 2625 μs 34944 μs 177169 μs 13001 μs As to losing credibility, I'm semi retired, I stopped trying to impress people a decade ago. If i were you I'd be more worried about your own image, bad mouthing other people's tech when you demonstrate you don't understand it hasn't made you like good to at least a few people here. It wouldn't be that hard to say "MDBM is great when used as an index into a DB, it works just fine for that. But you are going to have to rebuild the index after a power failure unless you take care to flush the data. It's somewhat unfair that the OP compared against my database because mine is slower because it handles crashes." Instead you come out with "MDBM is complete crap". Well, no, it's not. In the domain where it is useful it is actually quite useful, it's 10x faster for lookups than your DB. So the trade off is speed vs surviving reboots. For lots of people, speed is much more important. Machines don't crash every ten minutes. In fact, it's pretty common to see uptimes in 100's of days. Lets say that 100 days is average and lets say that it takes a full day to rebuild the MDBM. So MDBM is delivering 99/100 days of useful work. You deliver 100/100 days. Oh, wait, except that your useful work is running 10x slower if the DB is being used as an index. So you delivered 10/100 days. See why some people may prefer to use MDBM when it is put like that? Performance is a feature.

cbsmith 12y ago
I think the comparison was totally appropriate because the article specifically highlighted mdbm's design limitations. The whole point was that given the compromises mdbm makes, it is possible to yield significant performance gains, so if those design limitations fit your context (and it cited 4 specific examples where Yahoo uses them), you ought to consider it.