8 ms·
"For instance, on a cluster of five m3.large nodes, an even mixture of processes performing single-row inserts and selects over a few hundred rows pushed ~40 in
by BrentOzar 10y ago
"For instance, on a cluster of five m3.large nodes, an even mixture of processes performing single-row inserts and selects over a few hundred rows pushed ~40 inserts and ~20 reads per second (steady-state)."
Wow, that's...not fast.
- aphyr 10y agoI just want to emphasize that: a.) this is a beta product where the team has been focusing on completeness and correctness before performance b.) I was testing Cockroach from August through October--Cockroach Labs has put a good deal of work into performance in the last three months, but that work hasn't been merged into mainline yet. I expect things should be a good deal faster by 1.0. c.) Jepsen does some intentionally pathological things to databases. For instance, we choose extremely small Cockroach shard sizes to force constant rebalancing of the keyspace under inserts. Don't treat this as representative of a production workload.
- qaq 10y agoThey need to be at least 1000x to 10,000x faster on the read side though
- aphyr 10y agoI agree that it can and should be faster, but the targets will depend on what you're going for. 20k linearizable reads/sec on EBS-backed instances is, uh, maybe a tad optimistic.
- hinkley 10y agoI'm a little confused about how read speed is 2x slower than write speed. With respect to 'correctness', you're drifting into pyrrhic victory or 'not even wrong' territory at that point. When there are basic expectations of behavior that aren't being met, many of us would reject the idea that this code is 'correct'.
- aphyr 10y agoAs I mentioned in the article's introductory paragraphs, where Spanner forces a minimum latency on writes to ensure consistency, CockroachDB pushes that latency to reads which contend with a write.
- qaq 10y agoLatency is one thing throughput is a different animal 2 reads/second per m3.large node that really forces one to think there are some severe architectural issues.
- hinkley 10y agoYou're not wrong. Whoever downvoted you is pushing an agenda and I'm not happy about it. Reads per second is ultimately a measure of how many operations can be retired per second, not how long the operation takes. Anyone who has spent fifteen minutes learning about capacity planning should know that. 5 reads per second doesn't mean each read takes 200ms. It might mean 1 second per read spread across 5 threads of execution, or 3 seconds per read across 15 workers.
- stickfigure 10y agoThe parent poster (and many others in this thread) are assuming that performance under this deliberately pathological test is reflective of performance in the real world. Optimistic locking systems inherently perform poorly under contention. But they also perform better than pessimistic concurrency systems overall because in the real world we design applications to avoid contention. As an example, the Google App Engine datastore runs zillions of QPS across petabytes of data in a massive distributed cluster. But if you build an app that does nothing but mutate a single piece of state over and over, you'll top out at a couple transactions per second. This is painful if you're trying to build a simple counter, but with minimal care you can build a system that scales to any sized dataset and traffic volume.
- 10y ago
- sametmax 10y agoEven if they manage to multiply this by 100 on the final release, it's still way weaker than a regular sql db. I hope they have another selling point than performance.
- tormeh 10y agoIt's named cockroachDB...
- AsyncAwait 10y agoHorizontal scaleability might be one.
- chillydawg 10y agoBut why scale something when you can just run one postgres instance and provide 10000x the performance?
- sametmax 10y agoMy though exactly. It's already the case with faster DB. E.G: most projects my customers make me work on will never need scaling on multiple instances. They run really fast on a single postgres instance. One had some perf issues. First he just moved the DB to avoid having it on the same machine as the web server. Saved him a year. Then as the user base grew, he just bought a bigger server for the db. Right now it works really fast for 400k u/d, and there are still 2 offers of servers more powerful that the current one he has. By the time he needs to buy them, 2 new more incredible offers will probably exist. Which is a joke because more traffic means he has more money anyway. Having a cluster of nodes to scale is a lot of work. So unless you have 100 millions users doing crazy things, what's the point ?
- deleted 10y ago[deleted]
- threeseed 10y agoYou do understand that scalability is more than just concurrent users right ? Maybe not ? There are many very small startups doing work in IoT, Analytics, Social, Finance, Health etc who have ridiculously challenging storage needs. Trying to vertically scale is simply not an option when you have a 50 node Spark cluster or 1M IoT devices all streaming data into your database. Vertically scaling your DB doesn't work as it is largely an I/O challenge not a CPU/Storage one. But hey it's cute and lucky that you work on small projects. But many of us aren't and have to deal with these more exotic database solutions.
- bdarnell 10y ago(Cockroach Labs CTO here) Jepsen is essentially a worst-case stress test for a consistent database: all the transactions conflict with each other. CockroachDB's optimistic concurrency control performs worse with this kind of workload than the pessimistic (lock-based) concurrency control seen in most non-distributed databases. But this high-contention scenario is not the norm for most databases. Most transactions don't conflict with each other and can proceed without waiting. Without contention, CockroachDB can serve thousands of reads or writes per second per node (and depending on access patterns, can scale linearly with the number of nodes). And of course, we're continually working on performance (it's our main focus as we work towards 1.0), and things will get better on both the high- and low-contention scenarios. Just today we're about to land a major improvement for many high-contention tests, although it didn't speed up the jepsen tests as much as we had hoped.
- bfrog 10y agoCouldn't this be a problem for application authors and a potential source of DoS activities? It seems this should be a big red warning flag somewhere.
- stickfigure 10y agoNot really. All databases tend to struggle to under contention. If you are an application author building a high-scale app, you need to avoid contention by design. That single counter row you were planning to increment on every visit? Bad idea; maybe insert a new visit record instead. The contention in this example is an aspect of the test design, not database design. You wouldn't try to build an app this way.
- ej_campbell 10y agoIt'd only be a DOS vector, if the rest of the db bogged down during this type of contention.
- andymans 10y agoI can verify. I've been testing with node.js's pg client (not even using the 'native' option) and on 1 node I get > 1,000 single inserts /sec. I'm seeing this scale linearly too. NB I'm testing on a pretty overloaded macbook with only 8gb RAM. On a decent-sized cluster with production-grade hardware and network, performance is really unlikely to be a bottleneck. Also, it'll significantly improve during this year, one imagines.
- tveita 10y agoI wonder how the clock skew bounds affect performance. Would a more accurate clock increase throughput?
- aphyr 10y agoThe clock skew limit controls the maximum latency on reads--so yeah, if you're doing reads that contend with writes, improving clock resolution improves latency. Not entiiirely sure about the throughput impact though.
- tshannon 10y agoAt a greater risk of losing a server in your cluster due to automatic shutdown after too much clock-drift, right?
- jpgvm 10y agoYou wouldn't reduce the clock drift limit unless you had more accurate clocks however. If all of your servers are on bare metal and synced to a high resolution atomic or GPS clock then you can probably reduce it from 250ms to 10ms or less similar to Googles Spanner setup.