4 ms·
Does someone have any idea, how the benchmark results compare to other databases (e.g. redis) ?
by ma2rten 16y ago
Does someone have any idea, how the benchmark results compare to other databases (e.g. redis) ?
- rb2k_ 16y agoIt doesn't seem to have a network interface. It is somewhat comparable to InnoDB or Tokyo/Kyoto Cabinet
- davidhollander 16y agoBased on their numbers it looks slower than TokyoCabinet but still much faster than BerkleyDB. Would need to design a head to head benchmark. I don't think Redis is a good comparison as that's an in-memory database so better suited for the 10% of hot data you'd need to cache. Whereas disk stores like TokyoCabinet and LevelDB would be great for storing the other 90-100%. If your use case involves a large dataset and you don't have terabytes of RAM lying around, that is.
- sghemawat 16y agoOne of the leveldb authors here. TokyoCabinet is something we seriously considered using instead of writing leveldb. TokyoCabinet has great performance usually. I haven't done a careful head-to-head comparison, but it wouldn't surprise me if it was somewhat faster than leveldb for many workloads. Plus TokyoCabinet is more mature, has matching server code etc. and may therefore be a better fit for many projects. However because of a fundamental difference in data structures (TokyoCabinet uses btrees for ordered storage; leveldb uses log structured merge trees), random write performance (which is important for our needs) is significantly better in leveldb. This part we did measure. IIRC, we could fill TokyoCabinet with a million 100-byte writes in less than two seconds if writing sequentially, but the time ballooned to ~2000 seconds if we wrote randomly. The corresponding slowdown for leveldb is from ~1.5 seconds (sequential) to ~2.5 seconds (random).
- lisper 16y agoDid you consider fractal trees? http://tokutek.com/presentations/bender-Scalperf-9-09.pdf http://tokutek.com/presentations/bender-Scalperf-9-09.pdf
- psaccounts 16y agoIs there a more useful technical paper on Fractal Trees? Better yet is there a open source implementation of the same?
- jbapple 16y agoCache-oblivious streaming B-trees: http://supertech.csail.mit.edu/papers/sbtree.pdf http://supertech.csail.mit.edu/papers/sbtree.pdf I don't know for certain any open-source implementation, but I have heard COLAs are used in HBase.
- yingfeng 16y agofractal trees should have a faster read performance with leveldb with a comparable updating performance. There does not exist any open source solution for such structure, while we could hope that www.acunu.com might provide an open source solution for similary Stratified B-tree http://www.acunu.com/2011/04/log-file-systems-and-ssds-made-for-each-other/ http://www.acunu.com/2011/04/log-file-systems-and-ssds-made-...
- br1 16y agoBeing in-kernel, acunu is very far from the lightness of LevelDB.
- spullara 16y agoHave you looked at implementing CAS? It is the one thing missing for me to adopt it as a underlying store for my AvroBase API: http://www.javarants.com/2010/06/30/havrobase-a-searchable-evolvable-entity-store-on-top-of-hbase-and-solr/ http://www.javarants.com/2010/06/30/havrobase-a-searchable-e...
- davidhollander 16y ago> difference in data structures Always good to have more tools in one's arsenal in that case. I'll drop one of you a message if I write a Lua binding for it (using LuaJIT+embedded k/v for data services atm).
- cmartella 15y agoCould you please report about caching setup for random read test? was there any cache (a part of OS's) for either position/index lookup or payload? Or did you hit the disk without any caching of past operations? thanks