4 ms·

> Anyway, show me a database that passes Jepsen and I'll show you a database that only accepts writes to a single node. I know one - CockroachDB :). Relational

> Anyway, show me a database that passes Jepsen and I'll show you a database that only accepts writes to a single node. I know one - CockroachDB :). Relational, strongly-consistent, SQL, elastic, multi-active replication - the dream! (I work on it) https://www.cockroachlabs.com/blog/cockroachdb-beta-passes-jepsen-testing/ https://www.cockroachlabs.com/blog/cockroachdb-beta-passes-j... http://jepsen.io/analyses/cockroachdb-beta-20160829 http://jepsen.io/analyses/cockroachdb-beta-20160829

hmottestad 9y ago
RethinkDB also passes jepsen.
- linkmotif 9y ago
  It's also defunct so... :)
  - jjuel 9y ago
    Open Sourced is definitely different than defunct...
    - linkmotif 9y ago
      I know I know I was just kidding :D
    - manigandham 9y ago
      Without a company sponsoring and providing support, it is effectively dead for any serious commercial use.
- a-robinson 9y ago
  For single-key operations only, but that is still much better than the average system he's looked at.
linkmotif 9y ago
Hehe ;) Not to mention—it's Postgres compatible!!
StreamBright 9y ago
I am a little hesitant using a database that is younger than 10 years of age (same goes to programming languages, frameworks, etc.).
Thaxll 9y ago
“For instance, on a cluster of five m3.large nodes, an even mixture of processes performing single-row inserts and selects over a few hundred rows pushed ~40 inserts and ~20 reads per second (steady-state).” If someone can explains those abysmal low numbers ...
- andreimatei1 9y ago
  Well, those numbers, if I remember correctly, refer to insert rate in the face of some extreme amounts of contention - tons of overlapping reads and writes (that's how the test was trying to trigger consistency violations). And moreover, they were measured while the Jepsen test framework was messing with the cluster. Contention is a problem for every database, and particularly so for CRDB. In the absence of contention, we routinely see thousands of queries per second per node. Since the time of that analyses, we have done a significant amount of work for speeding up these high-contention scenarios, with quite dramatic differences in some cases we looked at. We pretty much changed our transaction execution model from a more "optimistic" one where transactions can abort each other and induce thrashing, to something resembling more the traditional row locks. So, hopefully, even for these atypical uses cases we should generally perform much better.
- AlisdairO 9y ago
  "These, of course, are pitiful numbers for a database. However, it’s important to note that these numbers are not representative of what real-world applications can expect to see from CockroachDB. First, this is a test in which the Jepsen nemesis is continuously tweaking the network configuration to cause (and heal) partitions between different nodes. You don’t commonly see behavior like this in a real world deployment. Second, tests like these deliberately try to reach high levels of contention, in which many transactions are trying to operate on the same keys. In real-world situations contention tends to be much lower as each user is operating mainly on their own data. The performance of all transactional systems suffer under high contention (although CockroachDB’s optimistic concurrency model fares worse than most)"
- arjunnarayan 9y ago
  Reposting my comment from when this came up on our 1.0 launch: IIRC, in the case that aphyr refers to for these specific numbers, the reads are scans that span multiple shards[1], while the writes are writes to single shards. [1] even though aphyr says it's just a hundred rows, the tables are split into multiple shards because aphyr in this case was specifically testing our correctness in multi-shard contention scenarios. In production you wouldn't have multi-shard reads crop up until you were doing scans for tables that were hundreds of thousands of rows in size[2]. It's easier to picture if you think of this performance speed difference in the scenario where you are doing full table scans spanning multiple shards on multiple computers while the underlying rows are being rapidly mutated by contending write transactions. The transactions get constantly aborted to avoid giving a non-serializable result, and performance is suffering. We agree that the numbers in this contention scenario are too low, and we are actively working on high-contention performance (and performance in general) leading up to our 1.0 release[3]. [2] Specifically, we break up into multiple shards when a single shard exceeds 64mb in size. [3] You can follow along on one of the PRs that address this specific performance issue here: https://github.com/cockroachdb/cockroach/pull/13501 https://github.com/cockroachdb/cockroach/pull/13501
didip 9y ago
CockroachDB peeps, you guys should really publish a new blog post with updated performance benchmarks. Super low throughput numbers is deterring a lot of potential users, imho.

hmottestad 9y ago

RethinkDB also passes jepsen.

linkmotif 9y ago
It's also defunct so... :)
- jjuel 9y ago
  Open Sourced is definitely different than defunct...
  - linkmotif 9y ago
    I know I know I was just kidding :D
  - manigandham 9y ago
    Without a company sponsoring and providing support, it is effectively dead for any serious commercial use.
a-robinson 9y ago
For single-key operations only, but that is still much better than the average system he's looked at.

linkmotif 9y ago

Hehe ;) Not to mention—it's Postgres compatible!!

StreamBright 9y ago

I am a little hesitant using a database that is younger than 10 years of age (same goes to programming languages, frameworks, etc.).

Thaxll 9y ago

“For instance, on a cluster of five m3.large nodes, an even mixture of processes performing single-row inserts and selects over a few hundred rows pushed ~40 inserts and ~20 reads per second (steady-state).” If someone can explains those abysmal low numbers ...

andreimatei1 9y ago
Well, those numbers, if I remember correctly, refer to insert rate in the face of some extreme amounts of contention - tons of overlapping reads and writes (that's how the test was trying to trigger consistency violations). And moreover, they were measured while the Jepsen test framework was messing with the cluster. Contention is a problem for every database, and particularly so for CRDB. In the absence of contention, we routinely see thousands of queries per second per node. Since the time of that analyses, we have done a significant amount of work for speeding up these high-contention scenarios, with quite dramatic differences in some cases we looked at. We pretty much changed our transaction execution model from a more "optimistic" one where transactions can abort each other and induce thrashing, to something resembling more the traditional row locks. So, hopefully, even for these atypical uses cases we should generally perform much better.
AlisdairO 9y ago
"These, of course, are pitiful numbers for a database. However, it’s important to note that these numbers are not representative of what real-world applications can expect to see from CockroachDB. First, this is a test in which the Jepsen nemesis is continuously tweaking the network configuration to cause (and heal) partitions between different nodes. You don’t commonly see behavior like this in a real world deployment. Second, tests like these deliberately try to reach high levels of contention, in which many transactions are trying to operate on the same keys. In real-world situations contention tends to be much lower as each user is operating mainly on their own data. The performance of all transactional systems suffer under high contention (although CockroachDB’s optimistic concurrency model fares worse than most)"
arjunnarayan 9y ago
Reposting my comment from when this came up on our 1.0 launch: IIRC, in the case that aphyr refers to for these specific numbers, the reads are scans that span multiple shards[1], while the writes are writes to single shards. [1] even though aphyr says it's just a hundred rows, the tables are split into multiple shards because aphyr in this case was specifically testing our correctness in multi-shard contention scenarios. In production you wouldn't have multi-shard reads crop up until you were doing scans for tables that were hundreds of thousands of rows in size[2]. It's easier to picture if you think of this performance speed difference in the scenario where you are doing full table scans spanning multiple shards on multiple computers while the underlying rows are being rapidly mutated by contending write transactions. The transactions get constantly aborted to avoid giving a non-serializable result, and performance is suffering. We agree that the numbers in this contention scenario are too low, and we are actively working on high-contention performance (and performance in general) leading up to our 1.0 release[3]. [2] Specifically, we break up into multiple shards when a single shard exceeds 64mb in size. [3] You can follow along on one of the PRs that address this specific performance issue here: https://github.com/cockroachdb/cockroach/pull/13501 https://github.com/cockroachdb/cockroach/pull/13501

didip 9y ago

CockroachDB peeps, you guys should really publish a new blog post with updated performance benchmarks. Super low throughput numbers is deterring a lot of potential users, imho.