6 ms·

That explains a lot, I had guessed as much. hyc_symas should give it a rest, mdbm isn't competing with an actual DB, it's a memory mapped hash, a fast one and

That explains a lot, I had guessed as much. hyc_symas should give it a rest, mdbm isn't competing with an actual DB, it's a memory mapped hash, a fast one and useful for a lot of things, but it's not a DB. I dunno where he got that idea from in the first place.

hyc_symas 12y ago
> I dunno where he got that idea from in the first place. You really started posting on this comment thread without reading the title of the article?
- luckydude 12y ago
  Lots of things are called a database without having database functionality. man gdbm: GDBM - The GNU database manager. So you are spouting all your disdain based on a title? Just curious, have you even run any version of mdbm? Or are you just spitting out baseless opinions? I'm sure your code is awesome and all but man, you are defensive. If your code is that great let it speak for itself. Bashing stuff that isn't even trying to compete with you makes you look pretty insecure. You remind me of me when I was younger; that's not a compliment, I was a raging asshole.
  - hyc_symas 12y ago
    Don't be silly. My opinions are never baseless. We used MDBM in OpenLDAP for a few years on SGI Irix. I haven't touched it myself in something like 9 or 10 years though. http://www.openldap.org/lists/openldap-devel/199903/msg00094.html http://www.openldap.org/lists/openldap-devel/199903/msg00094... http://www.openldap.org/lists/openldap-devel/200501/msg00053.html http://www.openldap.org/lists/openldap-devel/200501/msg00053... Using DBM-style DBs was an endless nightmare of corruption bugs. That technology belongs firmly in the distant past, we have better solutions today. http://www.openldap.org/lists/openldap-software/200607/msg00070.html http://www.openldap.org/lists/openldap-software/200607/msg00... I'm not bashing MDBM because it's a competitor; it obviously isn't a competitor. Heck the only reason I'm commenting in this thread is because you asked me to elaborate on its design flaws. Now you're hurt that I answered your question. Don't ask if you don't want to know the answer. ... seems like us folks in OpenLDAP weren't the only ones to have bad experiences with it. This guy in this same discussion seems to share the basic sentiment. https://news.ycombinator.com/item?id=8733819 https://news.ycombinator.com/item?id=8733819
    - luckydude 12y ago
      Where did I say your opinions are baseless? Edit. Huh, yeah I did. In the same place where I asked if you had ever used mdbm or are you spouting baseless opinions. So have you used it? All the links you provided talk about various DBM backends, not one of them point to mdbm as a problem. So again, have you used it or are you in fact stating baseless opinions? I'm sorry the openldap couldn't figure out how to use MDBM in a way that worked for them. Yahoo clearly has. We have. Others have. It works extremely well for what it was designed for and the perf numbers show that it spanks the living crap out of your stuff. But your stuff does more, which is cool. Why don't you focus on that rather than trying to crap all over some useful code? It's clearly not a threat to you. I don't see how the world is well served by your comments. For certain problems MDBM is way the hell better than what you have. It's a narrow niche, doesn't compete with you, so why all the fuss? As for me asking you to comment, yup, I did, but you were already well on your way of banging on technology you clearly didn't understand. I think I get why it didn't work for you, your comments have made it clear you have no idea how it works. I went and read all the links you provided above, not one mentioned mdbm having corruption bugs. It's entirely possible that the other DBM style dbs had bugs, or it is possible that people were inserting / deleting in a first / next loop, whatever. But pointing a finger at MDBM and claiming corruption bugs, how about you substantiate that claim? Seems somewhat flawed when multiple companies have used it for 10 years or more and it seems to work fine for them.
      - hyc_symas 12y ago
        You were already losing credibility due to your inattention to detail. I never mentioned performance anywhere in this thread. It's stupid to even talk about it since a hash doesn't do ordered lookups. But now that you're raising the topic: https://github.com/hyc/leveldb/tree/benches/doc/bench https://github.com/hyc/leveldb/tree/benches/doc/bench "Spanks the living crap" - must be some heretofore unknown definition of "spanks". ./db_bench_mdbm MDBM: version 4.11.1 Date: Mon Dec 15 04:39:20 2014 CPU: 4 * Intel(R) Core(TM)2 Extreme CPU Q9300 @ 2.53GHz CPUCache: 6144 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) ------------------------------------------------ fillrandsync : 40.627 micros/op 24614 ops/sec; 2.7 MB/s (1000 ops) 65604 /tmp/leveldbtest-1000 fillrandom : 16.356 micros/op 61137 ops/sec; 6.8 MB/s 122056 /tmp/leveldbtest-1000 fillrandbatch : 5.499 micros/op 181850 ops/sec; 20.1 MB/s 121936 /tmp/leveldbtest-1000 fillseqsync : 40.163 micros/op 24898 ops/sec; 2.8 MB/s (1000 ops) 65604 /tmp/leveldbtest-1000 fillseq : 16.424 micros/op 60886 ops/sec; 6.7 MB/s 175724 /tmp/leveldbtest-1000 fillseqbatch : 5.648 micros/op 177041 ops/sec; 19.6 MB/s 175724 /tmp/leveldbtest-1000 overwrite : 16.290 micros/op 61385 ops/sec; 6.8 MB/s 175724 /tmp/leveldbtest-1000 readrandom : 0.598 micros/op 1672444 ops/sec; (1000000 of 1000000 found) readseq : 0.096 micros/op 10370216 ops/sec; 1147.2 MB/s ./db_bench_mdb LMDB: version LMDB 0.9.14: (September 20, 2014) Date: Mon Dec 15 04:41:37 2014 CPU: 4 * Intel(R) Core(TM)2 Extreme CPU Q9300 @ 2.53GHz CPUCache: 6144 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) ------------------------------------------------ fillrandsync : 12.818 micros/op 78015 ops/sec; 8.6 MB/s (1000 ops) 224 /tmp/leveldbtest-1000/dbbench_mdb-1 224 /tmp/leveldbtest-1000 fillrandom : 4.275 micros/op 233923 ops/sec; 25.9 MB/s 116548 /tmp/leveldbtest-1000/dbbench_mdb-2 116548 /tmp/leveldbtest-1000 fillrandbatch : 3.490 micros/op 286502 ops/sec; 31.7 MB/s 126384 /tmp/leveldbtest-1000/dbbench_mdb-3 126384 /tmp/leveldbtest-1000 fillseqsync : 14.972 micros/op 66791 ops/sec; 7.4 MB/s (1000 ops) 172 /tmp/leveldbtest-1000/dbbench_mdb-4 172 /tmp/leveldbtest-1000 fillseq : 2.231 micros/op 448145 ops/sec; 49.6 MB/s 125872 /tmp/leveldbtest-1000/dbbench_mdb-5 125872 /tmp/leveldbtest-1000 fillseqbatch : 0.425 micros/op 2355457 ops/sec; 260.6 MB/s 125872 /tmp/leveldbtest-1000/dbbench_mdb-6 125872 /tmp/leveldbtest-1000 overwrite : 4.881 micros/op 204857 ops/sec; 22.7 MB/s 125872 /tmp/leveldbtest-1000/dbbench_mdb-6 125872 /tmp/leveldbtest-1000 readrandom : 1.166 micros/op 857624 ops/sec; (1000000 of 1000000 found) readseq : 0.059 micros/op 17092556 ops/sec; 1890.9 MB/s readreverse : 0.042 micros/op 23814626 ops/sec; 2634.5 MB/s Feel free to submit a patch if I got anything wrong in that driver; it was a pretty hasty patch. This is running on tmpfs, so no I/O involved. MDBM is faster on random read, which is what you'd expect since it's a hash and doesn't have to navigate down a tree to locate a record. Aside from that, it's pretty pedestrian.

hyc_symas 12y ago

> I dunno where he got that idea from in the first place. You really started posting on this comment thread without reading the title of the article?

luckydude 12y ago
Lots of things are called a database without having database functionality. man gdbm: GDBM - The GNU database manager. So you are spouting all your disdain based on a title? Just curious, have you even run any version of mdbm? Or are you just spitting out baseless opinions? I'm sure your code is awesome and all but man, you are defensive. If your code is that great let it speak for itself. Bashing stuff that isn't even trying to compete with you makes you look pretty insecure. You remind me of me when I was younger; that's not a compliment, I was a raging asshole.
- hyc_symas 12y ago
  Don't be silly. My opinions are never baseless. We used MDBM in OpenLDAP for a few years on SGI Irix. I haven't touched it myself in something like 9 or 10 years though. http://www.openldap.org/lists/openldap-devel/199903/msg00094.html http://www.openldap.org/lists/openldap-devel/199903/msg00094... http://www.openldap.org/lists/openldap-devel/200501/msg00053.html http://www.openldap.org/lists/openldap-devel/200501/msg00053... Using DBM-style DBs was an endless nightmare of corruption bugs. That technology belongs firmly in the distant past, we have better solutions today. http://www.openldap.org/lists/openldap-software/200607/msg00070.html http://www.openldap.org/lists/openldap-software/200607/msg00... I'm not bashing MDBM because it's a competitor; it obviously isn't a competitor. Heck the only reason I'm commenting in this thread is because you asked me to elaborate on its design flaws. Now you're hurt that I answered your question. Don't ask if you don't want to know the answer. ... seems like us folks in OpenLDAP weren't the only ones to have bad experiences with it. This guy in this same discussion seems to share the basic sentiment. https://news.ycombinator.com/item?id=8733819 https://news.ycombinator.com/item?id=8733819
  - luckydude 12y ago
    Where did I say your opinions are baseless? Edit. Huh, yeah I did. In the same place where I asked if you had ever used mdbm or are you spouting baseless opinions. So have you used it? All the links you provided talk about various DBM backends, not one of them point to mdbm as a problem. So again, have you used it or are you in fact stating baseless opinions? I'm sorry the openldap couldn't figure out how to use MDBM in a way that worked for them. Yahoo clearly has. We have. Others have. It works extremely well for what it was designed for and the perf numbers show that it spanks the living crap out of your stuff. But your stuff does more, which is cool. Why don't you focus on that rather than trying to crap all over some useful code? It's clearly not a threat to you. I don't see how the world is well served by your comments. For certain problems MDBM is way the hell better than what you have. It's a narrow niche, doesn't compete with you, so why all the fuss? As for me asking you to comment, yup, I did, but you were already well on your way of banging on technology you clearly didn't understand. I think I get why it didn't work for you, your comments have made it clear you have no idea how it works. I went and read all the links you provided above, not one mentioned mdbm having corruption bugs. It's entirely possible that the other DBM style dbs had bugs, or it is possible that people were inserting / deleting in a first / next loop, whatever. But pointing a finger at MDBM and claiming corruption bugs, how about you substantiate that claim? Seems somewhat flawed when multiple companies have used it for 10 years or more and it seems to work fine for them.
    - hyc_symas 12y ago
      You were already losing credibility due to your inattention to detail. I never mentioned performance anywhere in this thread. It's stupid to even talk about it since a hash doesn't do ordered lookups. But now that you're raising the topic: https://github.com/hyc/leveldb/tree/benches/doc/bench https://github.com/hyc/leveldb/tree/benches/doc/bench "Spanks the living crap" - must be some heretofore unknown definition of "spanks". ./db_bench_mdbm MDBM: version 4.11.1 Date: Mon Dec 15 04:39:20 2014 CPU: 4 * Intel(R) Core(TM)2 Extreme CPU Q9300 @ 2.53GHz CPUCache: 6144 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) ------------------------------------------------ fillrandsync : 40.627 micros/op 24614 ops/sec; 2.7 MB/s (1000 ops) 65604 /tmp/leveldbtest-1000 fillrandom : 16.356 micros/op 61137 ops/sec; 6.8 MB/s 122056 /tmp/leveldbtest-1000 fillrandbatch : 5.499 micros/op 181850 ops/sec; 20.1 MB/s 121936 /tmp/leveldbtest-1000 fillseqsync : 40.163 micros/op 24898 ops/sec; 2.8 MB/s (1000 ops) 65604 /tmp/leveldbtest-1000 fillseq : 16.424 micros/op 60886 ops/sec; 6.7 MB/s 175724 /tmp/leveldbtest-1000 fillseqbatch : 5.648 micros/op 177041 ops/sec; 19.6 MB/s 175724 /tmp/leveldbtest-1000 overwrite : 16.290 micros/op 61385 ops/sec; 6.8 MB/s 175724 /tmp/leveldbtest-1000 readrandom : 0.598 micros/op 1672444 ops/sec; (1000000 of 1000000 found) readseq : 0.096 micros/op 10370216 ops/sec; 1147.2 MB/s ./db_bench_mdb LMDB: version LMDB 0.9.14: (September 20, 2014) Date: Mon Dec 15 04:41:37 2014 CPU: 4 * Intel(R) Core(TM)2 Extreme CPU Q9300 @ 2.53GHz CPUCache: 6144 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) ------------------------------------------------ fillrandsync : 12.818 micros/op 78015 ops/sec; 8.6 MB/s (1000 ops) 224 /tmp/leveldbtest-1000/dbbench_mdb-1 224 /tmp/leveldbtest-1000 fillrandom : 4.275 micros/op 233923 ops/sec; 25.9 MB/s 116548 /tmp/leveldbtest-1000/dbbench_mdb-2 116548 /tmp/leveldbtest-1000 fillrandbatch : 3.490 micros/op 286502 ops/sec; 31.7 MB/s 126384 /tmp/leveldbtest-1000/dbbench_mdb-3 126384 /tmp/leveldbtest-1000 fillseqsync : 14.972 micros/op 66791 ops/sec; 7.4 MB/s (1000 ops) 172 /tmp/leveldbtest-1000/dbbench_mdb-4 172 /tmp/leveldbtest-1000 fillseq : 2.231 micros/op 448145 ops/sec; 49.6 MB/s 125872 /tmp/leveldbtest-1000/dbbench_mdb-5 125872 /tmp/leveldbtest-1000 fillseqbatch : 0.425 micros/op 2355457 ops/sec; 260.6 MB/s 125872 /tmp/leveldbtest-1000/dbbench_mdb-6 125872 /tmp/leveldbtest-1000 overwrite : 4.881 micros/op 204857 ops/sec; 22.7 MB/s 125872 /tmp/leveldbtest-1000/dbbench_mdb-6 125872 /tmp/leveldbtest-1000 readrandom : 1.166 micros/op 857624 ops/sec; (1000000 of 1000000 found) readseq : 0.059 micros/op 17092556 ops/sec; 1890.9 MB/s readreverse : 0.042 micros/op 23814626 ops/sec; 2634.5 MB/s Feel free to submit a patch if I got anything wrong in that driver; it was a pretty hasty patch. This is running on tmpfs, so no I/O involved. MDBM is faster on random read, which is what you'd expect since it's a hash and doesn't have to navigate down a tree to locate a record. Aside from that, it's pretty pedestrian.