7 ms·
20x times faster than BerkeleyDB is quite impresive. Would love to see a implementation of this.
by jburgueno 13y ago
20x times faster than BerkeleyDB is quite impresive. Would love to see a implementation of this.
- bigtones 13y agoYou will, it's built into the next version of SQL Server.
- alpb 13y agoUmm, how do you know? Do you work at Microsoft?
- snaky 13y agoBerkeleyDB is not so fast actually comparing to modern alternatives http://symas.com/mdb/microbench/ http://symas.com/mdb/microbench/
- jules 13y agoLies, damned lies, statistics, and benchmarks. Those benchmarks seem too good to be true. And reading the associated information they do look like that they are too good to be true. They claim zero-copy access to the database, which is great. But that probably means that in their read benchmarks, they are just returning a pointer to a record, and are probably not reading the actual record from disk (or for databases that live entirely in memory: into CPU cache). This gives an unfair view of the performance compared to databases that do read the data into memory (or CPU cache). While it's great that the database itself doesn't read the record, lets face it, most clients will need the actual record and not just a pointer to it. That is after all the point of a database. This also explains the unreal performance for large records. They do 30 million reads of 100 kilobyte records per second. If they were actually reading the records that would mean that their disk is doing 3 terabytes per second throughput. I want that disk!!! The hard disk and SSD also have exactly the same performance, so that means that they aren't even hitting the disk at all. So yes, they are cheating.
- easytiger 13y agoThe main issue with those stats is that they are comparing sqlite3 opened liked this: status = sqlite3_open(file_name, &db_); Writing to a file, with full ACID completeness and everything been put on this disk to some in memory key value stores. Completely different thing.
- snaky 13y ago>SQLite: We tuned SQLite's performance, by setting its locking mode to exclusive. We also enabled SQLite's write-ahead logging >Databases are opened in asynchronous write mode. (LevelDB's sync option, TreeDB's OAUTOSYNC option, SQLite3's synchronous options are all turned off, MDB uses the MDB_NOSYNC option, and BerkeleyDB uses the DB_TXN_WRITE_NOSYNC option). I.e., every write is pushed to the operating system, but the benchmark does not wait for the write to reach the disk. By the way there is SQLite with LMDB backend - https://gitorious.org/mdb/sqlightning https://gitorious.org/mdb/sqlightning >Using tool/speedtest.tcl in the SQLite source tree, the time to insert 1000 records on my laptop SSD was 22.42 seconds using the original code, and only 1.06 seconds using MDB. Both tests were run 3 times, with results average
- easytiger 13y agoYour point eludes me? For a start that is not tuning it. It is still using synchronous IO
- snaky 13y agoDid you look at http://symas.com/mdb/microbench/db_bench_sqlite3.cc http://symas.com/mdb/microbench/db_bench_sqlite3.cc? Search for 'write_sync'
- easytiger 13y agoAhh yea i missed, apologies std::string sync_stmt = (write_sync) ? "PRAGMA synchronous = FULL" : "PRAGMA synchronous = OFF"; Still. how is it a valid compare if it is getting it to ssd/hdd and the others are writing to memory? Just open sqlite3 with :memory:
- ldng 13y ago"MDB write performance when using its writable mmap option." Not really comparing apples, is it ? It doesn't seem fair to me.
- jules 13y agoWhy is that unfair?
- ldng 13y agoI did not delve into the details as you might have, but whenever I see mmap associated with huge perf I tend to wonder about corruption. I've seen to many bench comparing ondisk stores with in mmap stores flushed to disk one in a while. Then you need extra steps/configs to get good enough reliability that hinder perf.
- snaky 13y ago>whenever I see mmap associated with huge perf I tend to wonder about corruption If that was the case with lmdb implementation, that works as OpenLDAP backend all over the world, most probably you would have heard about it from the news.
- ldng 13y agoYou've have a point. But then, it's a bench mark and it would not be the first time that a bench is done using different settings than in production. Like I said, that lmdb is used in OpenLDAP doesn't automatically mean it is used with the exact same configuration as in the benchmark ;-) Nor did I say a memory store is bad, it's just you're not comparing the same things. Redis could be added to the bench for instance.
- hyc_symas 13y agoIMO main-memory stores are bad, for multiple reasons. LMDB is not a main memory store, it is a fully ACID-compliant transactional store.