5 ms·
10x insert performance improvement seems very small considering this simple benchmark someone did: http://stackoverflow.com/questions/15151554/comparing-mongod
by RyanZAG 13y ago
10x insert performance improvement seems very small considering this simple benchmark someone did:
http://stackoverflow.com/questions/15151554/comparing-mongodb-and-rethinkdb-bulk-insert-performance http://stackoverflow.com/questions/15151554/comparing-mongod...
MongoDB: 0m0.618s
RethinkDB: 2m2.502s
Although as stated in that post, this is likely because of the different fsync() policies between the two databases.
- coffeemug 13y agoThis is a bit misleading -- the performance difference between 1.6.1 and 1.7 ranges from 2x to >1000x depending on the circumstances (size of documents, durability, whether the client waits on network reply, etc.) This also doesn't account for 10-100x performance improvements on the batch insert algorithm made prior to 1.7. I suspect that RethinkDB 1.7 would be comparable to Mongo on insert performance workload described in this stack overflow thread.
- RyanZAG 13y agoIt was not deliberately misleading - it's simply the top result when doing a google search on rethinkdb and mongo performance. It's a shame that decent, general case database benchmarks are so hard to find.
- coffeemug 13y agoSorry, I meant to say that the benchmark in the release announcement is misleading (since it doesn't address all relevant issues), not the link in your comment. We're working on getting authoritative benchmarks out, but unfortunately good benchmarks are extremely time consuming (much like any science experiment).
- univerz 13y agoit would be great to see comparison with tokutek benchmark on mongodb / version with fractal tree indexes http://www.tokutek.com/2013/05/sysbench-benchmark-for-mongodb-v0-1-0-performance-update/ http://www.tokutek.com/2013/05/sysbench-benchmark-for-mongod...
- hosay123 13y agoAccording to that page, the Mongo test was using unacknowledged writes, so not exactly a useful comparison
- jamesaguilar 13y ago/dev/null: 0m0.001s /dev/null is clearly the best database. (If your performance numbers are too good to be true, they might not be true.)
- alexpopescu 13y agoOne aspect of micro-benchmarks that is most of the time ignored is that they reveal different default settings various systems come with. And unfortunately after seeing the results, not too many dig deeper to figure them out.
- jamesaguilar 13y agoIndeed. And the default settings reveal something about a system's priorities. If you care about your users' data, then Mongo's default settings should be a giant red flag. The revealed priority is seeming fast.
- amenod 13y agoIn noSQL world, any DB's default settings should be a giant red flag. Don't get me started about HBase... I lost 2 hours of data this way. :(
- coffeemug 13y agoThat was actually one of the explicit design goals of Rethink -- pick defaults such that users never have to wonder about the safety of their data. I know the folks at Riak are also in this camp, so there are definitely NoSQL dbs that do this well.
- shin_lao 13y agoI'm not sure Riak is a good example as it's another example of an extremely slow database. There's also the angle that if it's to offer no performance benefit, perhaps a classic relational database will do it? I'm curious to hear your thoughts about it.
- Erwin 13y agoWell, on a PG 9.1.9 instance, which has fsync=on and an older Intel i7 on an Intel SSD, my 8-process program inserting a 120-byte TEXT value into a (SERIAL, TEXT) table, with transactions each doing 10 inserts, manages to finish inserting 100,000 groups of 10 "documents" in 37 seconds. So that's roughly 27,000 documents inserted per second (not sure if they are measuring individual inserts documents or groups of 10). According to iostat, this does 5000 write transactions per second to the disk. If you are logging throw-away data, you can use an UNLOGGED table. This brings it to 49,000 inserted rows per second. If you don't care about your data integrity, and set fsync=off, it goes up to 65,000 and seems to be limited by CPU.
- coffeemug 13y agoVery cool, thanks for the numbers! A few bits of info on the benchmark in the post. It uses a single process/thread/connection. Adding more concurrent clients would improve performance, though there is still some scalability work to be done. Ops/second in the graphs measures number of documents, not groups. One fundamental aspect of the benchmark is that it uses random keys, which is significantly slower (in any database) than using sequential inserts. I'm not sure how you insert into Postgres wrt to keys. I suspect postgres has better latency, and since the benchmark in the post is very much latency bound, this would account for the factor of four performance difference. It's something we still need to address. EDIT: 1 million documents / 37 seconds / 8 connections = 3378 ops/second, which is about a factor of four off from 800 ops/second that RethinkDB does on this. (This is obviously back of the napkin math.) There's lots of work to be done, I'm very much looking forward to publishing authoritative, scientific comparison benchmarks.
- malkia 13y agoFor postgres I've been using several threads, each with its own connection, and all going through the same transaction (using the number exported by pg_export_snapshot()) - with this the speedup is quite big (c++ project using libpq). I expect that if I move to binary format for inserting it might be even faster. I'm basing this since my reader is binary and that made x3-x4 improvements for my data set/computer/etc.
- orthecreedence 13y agoOk, now try doing 1000 inserts at once on Mongo. OH WAIT, global write lock.