6 ms·

Anyone has some numbers on performance? I tried RethinkDB 1.x and the performance wasn't quite there yet, specially bulk import and aggregations.

by mping 12y ago

coffeemug 12y ago
We'll be publishing a performance report soon (we didn't manage to get it out today). Rough numbers you can expect for 1KB size documents, 25M document database: 40K reads/sec/server, 5K writes/sec/server, roughly linear scalability across nodes. We should be able to get the report out in a couple of days.
- GordyMD 12y ago
  Looking forward to seeing the report!
- lobster_johnson 12y ago
  Any work done in 2.0 for improving aggregation performance? The last time I tried with 1.16, I gave up my testing when even the simplest aggregation query (count + group by with what should be a sequential, streaming scan) took literally minutes with RethinkDB, compared to <1s with PostgreSQL. Rethink coredumped before I gave it enough RAM, after which it blew up to around 7GB, whereas Postgres uses virtually no RAM, mostly OS buffers.
  - danielmewes 12y ago
    We did a couple of scalability improvements in 2.0, but didn't optimize groups and counts specifically. Would you mind writing me an email with your query or opening an issue at https://github.com/rethinkdb/rethinkdb/issues https://github.com/rethinkdb/rethinkdb/issues (unless you have already?)? I'd like to look into it to see how we can best improve this. We're planning to implemented a faster count algorithm that might help with this (https://github.com/rethinkdb/rethinkdb/issues/3949 https://github.com/rethinkdb/rethinkdb/issues/3949), but it's not completely trivial and will take us slightly longer to implement.
    - lobster_johnson 12y ago
      What I was doing is so trivial, you don't really need this information. This was my reference SQL query: select path, count(*) from posts group by path; (I don't have the exact Rethink query written down, but it was analogous to the SQL version.) You can demonstrate RethinkDB's performance issue with any largeish dataset by trying to group on a single field. The path column in this case has a cardinality of 94, and the whole dataset is about 1 million documents. Some rows are big, some not; each has metadata plus a JSON document. The Postgres table is around 3.1GB (1GB for the main table + a 2.1GB TOAST table). Postgres does a seqscan + hash aggregate in about 1500ms. It's been months since I did this, and I've since deleted RethinkDB and my test dataset.
      - SamReidHughes 12y ago
        Are you sure the analogous RethinkDB query was using the index? Iirc it's not enough just to use the column name (or wasn't, I don't keep up).
        lobster_johnson 12y ago
        It wasn't using an index, but then Postgres wasn't, either. I don't think aggregating via B-tree index is a good idea; aggregation is inherently suited to sequential access. An index is useful only when the selectivity is very low.
        SamReidHughes 12y ago
        If you wrote your query with group and count, with no index, then there would be problems with the performance. RethinkDB generally does not do query optimization, except in specific ways (mostly about distributing where the query is run), unless that's changed very recently. You can write that query so that it executes with appropriate memory usage with a map and reduce operation.
        lobster_johnson 12y ago
- javajosh 12y ago
  Thank you for providing some quick back-of-the-envelope numbers here, which is exactly what most people are looking for at a first pass. One question though - do those numbers change considerably between disk vs SSD?
  - coffeemug 12y ago
    The numbers were measured on SSD. If the active/hot dataset fits into RAM, the numbers between SSD and rotational don't change much. If the active dataset doesn't fit into RAM, RethinkDB performs significantly worse on rotational.
- cachvico 12y ago
  Can you give a ballpark on how many nodes Rethink can scale up to, and any future roadmap in that direction? Thank you for the fantastic product, by the way! :)
  - coffeemug 12y ago
    We often run tests scaling up to ~40 nodes without problems. You could probably push Rethink quite further than that, but I think over 100 nodes would be hard. The goal is to keep pushing the boundary indefinitely.
    - cachvico 12y ago
      Thank you Slava. Put all these numbers on the front of your webpage :)
- e12e 12y ago
  Rough numbers indeed - you forgot to define what a "server" is -- dedicated hw 16 core xeon with 4xssd in hw raid0 or a Digital Ocean vps with 512MB ram? ;-)
  - danielmewes 12y ago
    Daniel @ RethinkDB here. We'll release the details shortly. This was running on 12 core Xeon servers with 2 SSDs each in software RAID 0. There were also additional read queries running at the same time as the write queries, and the read throughput that coffeemug posted is the sustainable increase in reads/s that you get when adding an additional server to a cluster. Single-server performance is much higher due to missing network / message encoding overhead. I realize these numbers alone are still not very meaningful and there are many remaining questions (size and structure of the data set, exact queries performed etc). Rest assured that all of these details will be mentioned in the actual performance report that should be up soon.
    - e12e 12y ago
      10 gbps ethernet?
      - danielmewes 12y ago
        Two 1 GBit ethernet ports, one used for intra-cluster and one for client connections.
jfolkins 12y ago
I contributed the benchmarks to Dan's gorethink driver. Dan is great to collaborate with so if you want to hack on Go and contribute to OSS, consider giving his project a look. One way to improve writes is to batch them, an example is here. https://github.com/dancannon/gorethink/blob/master/benchmarks_test.go#L15-L43 https://github.com/dancannon/gorethink/blob/master/benchmark... I believe rethinkdb docs state that 200 is the optimum batch size. Another way is to enable the soft durability mode. http://rethinkdb.com/api/javascript/insert/ http://rethinkdb.com/api/javascript/insert/ "In soft durability mode RethinkDB will acknowledge the write immediately after receiving and caching it, but before the write has been committed to disk." https://github.com/dancannon/gorethink/blob/master/benchmarks_test.go#L79-L114 https://github.com/dancannon/gorethink/blob/master/benchmark... Obviously your business requirements come into play. I prefer the Hard writes because my data is important to me but I do insert debug messages using soft writes in one application I have. *Edit: Heh I forgot to mention, on my Macbook Pro I was getting 20k w/s while batching and using soft writes. Individual writes for me are hovering around 10k w/s on the 8 cpu 24gb instance i have. But yeah, define your business reqs then write your own benchmarks and see if the need is met. Many devs write benchmarks in order to be the fastest and not the correctest. Super lame.

Anyone has some numbers on performance? I tried RethinkDB 1.x and the performance wasn't quite there yet, specially bulk import and aggregations.

by mping 12y ago

Anyone has some numbers on performance? I tried RethinkDB 1.x and the performance wasn't quite there yet, specially bulk import and aggregations.

coffeemug 12y ago

We'll be publishing a performance report soon (we didn't manage to get it out today). Rough numbers you can expect for 1KB size documents, 25M document database: 40K reads/sec/server, 5K writes/sec/server, roughly linear scalability across nodes. We should be able to get the report out in a couple of days.

GordyMD 12y ago
Looking forward to seeing the report!
lobster_johnson 12y ago
Any work done in 2.0 for improving aggregation performance? The last time I tried with 1.16, I gave up my testing when even the simplest aggregation query (count + group by with what should be a sequential, streaming scan) took literally minutes with RethinkDB, compared to <1s with PostgreSQL. Rethink coredumped before I gave it enough RAM, after which it blew up to around 7GB, whereas Postgres uses virtually no RAM, mostly OS buffers.
- danielmewes 12y ago
  We did a couple of scalability improvements in 2.0, but didn't optimize groups and counts specifically. Would you mind writing me an email with your query or opening an issue at https://github.com/rethinkdb/rethinkdb/issues https://github.com/rethinkdb/rethinkdb/issues (unless you have already?)? I'd like to look into it to see how we can best improve this. We're planning to implemented a faster count algorithm that might help with this (https://github.com/rethinkdb/rethinkdb/issues/3949 https://github.com/rethinkdb/rethinkdb/issues/3949), but it's not completely trivial and will take us slightly longer to implement.
  - lobster_johnson 12y ago
    What I was doing is so trivial, you don't really need this information. This was my reference SQL query: select path, count(*) from posts group by path; (I don't have the exact Rethink query written down, but it was analogous to the SQL version.) You can demonstrate RethinkDB's performance issue with any largeish dataset by trying to group on a single field. The path column in this case has a cardinality of 94, and the whole dataset is about 1 million documents. Some rows are big, some not; each has metadata plus a JSON document. The Postgres table is around 3.1GB (1GB for the main table + a 2.1GB TOAST table). Postgres does a seqscan + hash aggregate in about 1500ms. It's been months since I did this, and I've since deleted RethinkDB and my test dataset.
    - SamReidHughes 12y ago
      Are you sure the analogous RethinkDB query was using the index? Iirc it's not enough just to use the column name (or wasn't, I don't keep up).
      - lobster_johnson 12y ago
        It wasn't using an index, but then Postgres wasn't, either. I don't think aggregating via B-tree index is a good idea; aggregation is inherently suited to sequential access. An index is useful only when the selectivity is very low.
        SamReidHughes 12y ago
        If you wrote your query with group and count, with no index, then there would be problems with the performance. RethinkDB generally does not do query optimization, except in specific ways (mostly about distributing where the query is run), unless that's changed very recently. You can write that query so that it executes with appropriate memory usage with a map and reduce operation.
        lobster_johnson 12y ago
javajosh 12y ago
Thank you for providing some quick back-of-the-envelope numbers here, which is exactly what most people are looking for at a first pass. One question though - do those numbers change considerably between disk vs SSD?
- coffeemug 12y ago
  The numbers were measured on SSD. If the active/hot dataset fits into RAM, the numbers between SSD and rotational don't change much. If the active dataset doesn't fit into RAM, RethinkDB performs significantly worse on rotational.
cachvico 12y ago
Can you give a ballpark on how many nodes Rethink can scale up to, and any future roadmap in that direction? Thank you for the fantastic product, by the way! :)
- coffeemug 12y ago
  We often run tests scaling up to ~40 nodes without problems. You could probably push Rethink quite further than that, but I think over 100 nodes would be hard. The goal is to keep pushing the boundary indefinitely.
  - cachvico 12y ago
    Thank you Slava. Put all these numbers on the front of your webpage :)
e12e 12y ago
Rough numbers indeed - you forgot to define what a "server" is -- dedicated hw 16 core xeon with 4xssd in hw raid0 or a Digital Ocean vps with 512MB ram? ;-)
- danielmewes 12y ago
  Daniel @ RethinkDB here. We'll release the details shortly. This was running on 12 core Xeon servers with 2 SSDs each in software RAID 0. There were also additional read queries running at the same time as the write queries, and the read throughput that coffeemug posted is the sustainable increase in reads/s that you get when adding an additional server to a cluster. Single-server performance is much higher due to missing network / message encoding overhead. I realize these numbers alone are still not very meaningful and there are many remaining questions (size and structure of the data set, exact queries performed etc). Rest assured that all of these details will be mentioned in the actual performance report that should be up soon.
  - e12e 12y ago
    10 gbps ethernet?
    - danielmewes 12y ago
      Two 1 GBit ethernet ports, one used for intra-cluster and one for client connections.

jfolkins 12y ago

I contributed the benchmarks to Dan's gorethink driver. Dan is great to collaborate with so if you want to hack on Go and contribute to OSS, consider giving his project a look. One way to improve writes is to batch them, an example is here. https://github.com/dancannon/gorethink/blob/master/benchmarks_test.go#L15-L43 https://github.com/dancannon/gorethink/blob/master/benchmark... I believe rethinkdb docs state that 200 is the optimum batch size. Another way is to enable the soft durability mode. http://rethinkdb.com/api/javascript/insert/ http://rethinkdb.com/api/javascript/insert/ "In soft durability mode RethinkDB will acknowledge the write immediately after receiving and caching it, but before the write has been committed to disk." https://github.com/dancannon/gorethink/blob/master/benchmarks_test.go#L79-L114 https://github.com/dancannon/gorethink/blob/master/benchmark... Obviously your business requirements come into play. I prefer the Hard writes because my data is important to me but I do insert debug messages using soft writes in one application I have. *Edit: Heh I forgot to mention, on my Macbook Pro I was getting 20k w/s while batching and using soft writes. Individual writes for me are hovering around 10k w/s on the 8 cpu 24gb instance i have. But yeah, define your business reqs then write your own benchmarks and see if the need is met. Many devs write benchmarks in order to be the fastest and not the correctest. Super lame.