3 ms·

Can anyone speak about the benchmark posted on the blog post? Is there some fundamental difference to CockroachDB that is causing it to be that much slower? Or

by Nican 7y ago

Can anyone speak about the benchmark posted on the blog post? Is there some fundamental difference to CockroachDB that is causing it to be that much slower? Or is it really just raw performance? EDIT: As another note, I believe Aurora added multi-master support recently. How does that compare as well?

rkarthik007 7y ago
Hi @Nican, sure thing. There are two benchmark workloads, simple inserts and secondary index. The simple inserts workload 50M unique key-values into the database using prepare-bind INSERT statements with 256 writer threads running in parallel. There were no reads against the database during this period. The secondary index inserts does the same things against a table which has an index on the value column (forcing each operation to transactionally update the primary and index tables). We use this simple workload frequently to test scalability of YugabyteDB. If interested, here is the sample app repo: https://github.com/YugaByte/yb-sample-apps https://github.com/YugaByte/yb-sample-apps Here is a previous version of the benchmark comparing YCQL (no YSQL here, it was not GA then), it has more details about the scenario we are going for (this part applies to this benchmark also): https://blog.yugabyte.com/yugabyte-db-vs-cockroachdb-performance-benchmarks-for-internet-scale-transactional-workloads/ https://blog.yugabyte.com/yugabyte-db-vs-cockroachdb-perform... The above post goes through more workloads than this one - we have not had the time to run everything on YSQL yet. The aim of this post was mainly to explore write perf and scalability vs Amazon Aurora. > As another note, I believe Aurora added multi-master support recently. How does that compare as well? Good question. Multi-master deployment sacrifices consistency with last writer wins semantics, so we did not benchmark against that. For example, the benchmark driver can write to one node and read from another before the data was replicated. But may still be interesting from a tradeoff perspective (perf vs consistency). Also, note that we have also just announced multi-master support between separate YugabyteDB cluster - so a benchmark is probably something we should do at some point anyway!
mikesun 7y ago
Benchmark ran on each DB’s default isolation mode which are different: serializable for Cockroach and snapshot for YugaByte. https://twitter.com/karthikr/status/1174031604106153984 https://twitter.com/karthikr/status/1174031604106153984

by Nican 7y ago

rkarthik007 7y ago

Hi @Nican, sure thing. There are two benchmark workloads, simple inserts and secondary index. The simple inserts workload 50M unique key-values into the database using prepare-bind INSERT statements with 256 writer threads running in parallel. There were no reads against the database during this period. The secondary index inserts does the same things against a table which has an index on the value column (forcing each operation to transactionally update the primary and index tables). We use this simple workload frequently to test scalability of YugabyteDB. If interested, here is the sample app repo: https://github.com/YugaByte/yb-sample-apps https://github.com/YugaByte/yb-sample-apps Here is a previous version of the benchmark comparing YCQL (no YSQL here, it was not GA then), it has more details about the scenario we are going for (this part applies to this benchmark also): https://blog.yugabyte.com/yugabyte-db-vs-cockroachdb-performance-benchmarks-for-internet-scale-transactional-workloads/ https://blog.yugabyte.com/yugabyte-db-vs-cockroachdb-perform... The above post goes through more workloads than this one - we have not had the time to run everything on YSQL yet. The aim of this post was mainly to explore write perf and scalability vs Amazon Aurora. > As another note, I believe Aurora added multi-master support recently. How does that compare as well? Good question. Multi-master deployment sacrifices consistency with last writer wins semantics, so we did not benchmark against that. For example, the benchmark driver can write to one node and read from another before the data was replicated. But may still be interesting from a tradeoff perspective (perf vs consistency). Also, note that we have also just announced multi-master support between separate YugabyteDB cluster - so a benchmark is probably something we should do at some point anyway!

mikesun 7y ago

Benchmark ran on each DB’s default isolation mode which are different: serializable for Cockroach and snapshot for YugaByte. https://twitter.com/karthikr/status/1174031604106153984 https://twitter.com/karthikr/status/1174031604106153984