7 ms·
Progress in performance and scalability with CockroachDB
- nawfalhasan 8y agoI'm totally new to cockroach so I have 2 questions.. 1. Is there a managed service of this db where it auto scales, does geo replication etc all by itself? 2. Is there any really good book on cockroachdb?
- orangechairs 8y agoWe released a managed version at the end of October, with auto-scaling, geo-replication, etc. --> https://www.cockroachlabs.com/product/managed/ https://www.cockroachlabs.com/product/managed/ Not sure that there any books on it yet.
- manigandham 8y agoThe managed service doesn't autoscale, it's provisioned capacity by cores. We just did a call about it.
- orangechairs 8y agoOur managed service is currently provisioned by cores. We automatically add nodes to your cluster based on your usage. You can also request to add more nodes if you anticipate spikes.
- nawfalhasan 8y agoThank you..
- gigatexal 8y agoI’m currently evaluating this as an alternative to vitess + percona mysql. But strict seralizability has its limitations.
- segmondy 8y agoWhy as an alternative to vitess? Are you hitting limits with vitess?
- gigatexal 8y agoBecause I believe the overhead to not be worth it. I’d rather a database be cloud native and have things like sharding and scaling built in and the coordination of that built in. I grok the CockroachDB way of doing things much more than how Vitess is and shocking fact most companies aren’t Google in size and don’t really need a Vitess or CockroachDB but get swept up in the Cloud,Cloud,Cloud! craze. The seemingly brittle nature of vtablets, vtgates, etc versus something built in is valuable to me.
- eloff 8y agoWhat do you mean "strict seralizability has its limitations", you need something stricter? Or you have a need for something weaker for some reason?
- gigatexal 8y agoIn the same way single threaded has limitations.
- evrydayhustling 8y agoThis is a crazy multiple! Anyone from the Cockroach team up for sharing what the key innovations were that are driving the improved performance?
- awoods187 8y agoI'm the author. We've introduced transactional write pipelining (covered in a forthcoming blog post), load-aware rebalancing, and completed general performance tuning which all contribute to our improved performance numbers.
- anticensor 8y agoSomeone should create a fork with work-safe name. CockroachDB brings connotation of cockroaches, who are known by eating almost everything and living almost everywhere.
- deleted 8y ago[deleted]
- geoka9 8y ago> known by eating almost everything and living almost everywhere Good attributes for a DB though?
- drodgers 8y agoThat's the idea though — your data will be hard to kill.
- deleted 8y ago[deleted]
- andreimatei1 8y agohttps://github.com/tbg/bikesheddb https://github.com/tbg/bikesheddb
- danellis 8y agoI have to admit, the name does set me on edge slightly. shudder
- ofrzeta 8y agoIn their presentation they have it abbreviated as CRDB, so that might be a viable direction without pain.
- deleted 8y ago[deleted]
- mlevental 8y agoyou know what's ironic? that in every thread there's one of you people complaining about the name - you're all just like cockroaches! no matter how successful cockroachdb becomes, no matter how technically impressive the product becomes, the naysayers never die. can you imagine 20 years ago someone complaining that google wasn't safe for work because it had a silly name? newsflash dummy: stop saying/thinking/repeating stupid things like this and it'll stop being the case that everyone is so conservative that silly names are inadmissible.
- brod 8y agoI've just released a small product using CockroachDB, in retrospect it was probably my favourite technical decision. Previously I'd used it as a toy and tested deployment strats but was skeptical (new tech and all that), but now that it's ticking along in the wild I'm very impressed across the board.
- qaq 8y agoNow we just need some benchmarks on a reasonable size dataset like 100TB and up
- nawfalhasan 8y agoNot an easy benchmark..
- qaq 8y agoOne would imaging they are testing with much larger datasets internally.
- Confiks 8y agoI was wondering, quite unrelated to the article, if anyone knows if CockroachDB would be suited for small databases (and comparably modest computing/memory resources). I very much like its distributed properties, but only have a simple table of usernames and corresponding cryptographic material. Is CRDB easy to run and manage?
- tshannon 8y agoI would expect your use case would be better suited by use just using postgres. However if you do need to scale to the point where you'd need to distribute your database and take advantage of CRDB's capabilities, it uses the Postgres protocol, so you most likely can just migrate your data and use the same code.
- andreimatei1 8y agoWe have clients using CRDB in pretty constrained environments, and they use it primarily because of the easy administration. I think you'll find it easier to use than a MySQL or Postgres, for example.
- manigandham 8y agoSqlite would be my first recommendation, unless you need client/server access.
- yellowapple 8y agoI think the GP's stated need for replication would preclude SQLite unless one's willing to write one's own replication system.
- manigandham 8y agoWhere's the stated need for replication?
- yellowapple 8y ago"I very much like its distributed properties"
- continuations 8y agoA few questions: 1) >631851 tpmC How many servers are needed to achieve this throughput? 2) >4 terabytes of unreplicated, frequently accessed data 4TB unreplicated data? Does that mean if a single node goes down you'll lose data (EDIT: I meant losing availability, not data)? That kinda ruins the whole point of having a distributed database. 3) If I'm reading the KV benchmaks correctly, it takes 5 nodes to achieve 100k tpm. That's 20k tpm per node. That's 333 tps per node. This is a 95% point read benchmark. Why is the tps (333 tps) so low? Is that normal? 4) How does CockroachDB compare to other distributed databases such as TiDB, FoundationDB, ScyllaDB?
- manigandham 8y agoScyllaDB is still the fastest at a key/value workload with per-query consistency settings and quorum reads/writes across multiple regions. If you need high-performance and low-latency, ScyllaDB wins. They are close to v3.0 which will have global secondary indexes and materialized views to improve data model flexibility. FoundationDB is also key/value but much lower-level and well proven for reliability. Don't have much experience with it and the latest release just introduced multi-regional capabilities, but the general tooling and documentation is still rough and it would take more effort to build a higher-level querying layer or client library. TiDB is interesting, but missing more features from MySQL than CRDB is missing from PostgreSQL, so it's effective if you want sharding on mysql but will need a few more releases before it gets polished. Vitess and Citus are good options if you just want sharding on top of existing mysql or postgres with full query support within a shard. There's also Yugabyte which is multi-modal Redis/Cassandra/SQL offering with multi-regional capabilities. CRDB is a great product with some of the easiest operations (although key management is a nightmare that they do not have a good plan for). It's fast enough for point-lookups and makes it easy to distribute and replicate your data across zones and regions. All nodes are part of a single cluster so read and write latencies will be high for global deployment, with the enterprise version having a workaround for local regional reads using pinned covering indexes. That works, but further lowers write performance. It also has trouble with large transactions and the middle ground between OLTP and OLAP with heavy joins. Good choice if you need easy scalability and SQL interface over performance and complex queries.
- qaq 8y agoAt what point is it cost effective to run CRDB vs PostgreSQL?