7 ms·
Look up MDBM. I spent a lot of time with a logic analyzer watching the cache misses go across the bus. I'd be pretty surprised if someone has done better. I c
by luckydude 9y ago
Look up MDBM. I spent a lot of time with a logic analyzer watching the cache misses go across the bus. I'd be pretty surprised if someone has done better.
I can find the code and repost it.
- dryman 9y agoThanks for the reference. I'm collecting a list of embedded key-value store to benchmark against. I'll tryout with this one first!
- corysama 9y agoFun note: Because of this line in the article > Because MDBM gives you raw pointers into the DB’s data, you have to be very careful about making sure you don’t have array over-runs, invalid pointer access, or the like. Someone wrote a Rust binding http://erickt.github.io/blog/2014/12/13/rust-and-mdbm/ http://erickt.github.io/blog/2014/12/13/rust-and-mdbm/
- dryman 9y agoWow that's a very interesting post! Maybe I should try to bring a rust binding to my project as well..
- posnet 9y agohttps://github.com/pmwkaa/engine.so https://github.com/pmwkaa/engine.so
- dom0 9y agoLMDB
- hyc_symas 9y agoI would expect this hash table to be much faster than LMDB for tables that fit in RAM. It doesn't have transactions or any kind of concurrent access guarantees so naturally it has lower access overhead. For data sets larger than RAM, or in multithreaded environments, it will be pretty unusable.
- dryman 9y agoI found some performance numbers from Yahoo. https://yahooeng.tumblr.com/post/104861108931/mdbm-high-speed-database https://yahooeng.tumblr.com/post/104861108931/mdbm-high-spee... It has random read time 0.45 μs. On my late 2013 iMac, i5 cpu, I got 0.089 μs random read. I yet to have a benchmark that measures sequential read, but the API supports it.
- searchfaster 9y ago> I can find the code and repost it. Here it is.. https://github.com/yahoo/mdbm https://github.com/yahoo/mdbm
- luckydude 9y agoSo Yahoo took the code and did all sorts of stuff to it. I've got the code pre-yahoo and that's the stuff that is fast. It's part of BitKeeper which is open source.
- tawayway 9y agoOut of curiosity, what kind of logic analyzer are you using that can see events that fast?
- luckydude 9y agoI dunno, it was a long time ago at SGI, maybe 23 years ago? The CPUs were 200mhz MIPS chips. One of the hardware guys heard me talking about trying to optimize this code and he set up the analyzer and taught me how to use it.