5 ms·
Ask HN: What has been your experience with CockroachDB?
I literally cannot find a single person describing their experience using CockroachDB, which is bizarre considering how long they have been around.
Would love to hear from someone who've used them in production.
- truth_seeker 4y agoWhy don't you try swapping your existing PG server with CockroachDB and see how it goes. Being a die hard fan of Postgresql for more than 15 years, I think time has come for me to select CockroachDB as new default. I miss Table Partitioning (only available in enterprise edition ) and Full text search (yet to be built). Also minimum hardware I had to employ was quad core and 8 gigs of RAM. But other than that everything is so smooth including installation, online documentation. It made me lazy.
- sean0- 4y agoregarding FTS: https://github.com/cockroachdb/cockroach/pull/85185 https://github.com/cockroachdb/cockroach/pull/85185 just sayin'.
- com 4y agoI know of two scaleups that have left it. As usual, this might just be poor implementation choices rather than product or support issues.
- notmymainlogin 4y ago(note the username!) Check out Aerospike. It's the most robust and truly "zero effort scale" real time data platform I've ever seen. Properly configured and used, it whups everything in terms of TPS and performance/machine aka TCO. By a huge, huge, factor. Downside is ... everything else. :/ Less polished developer experience, docs, &c. There are so many huge customers, but very few who are talking about it. Requires more of a time investment -- because you're learning a new platform, not using a drop in replacement for what you already know. But the reward is performance beyond anything else. (the VLDB 2016 paper is a good high level overview of the architecture) If you do check it out, be sure to use version 6.1 or later, in particular see what can be done combining secondary indexes with container data types.... A, uh, birdie told me that. :P edit: Same birdie told me Aerospike has customers who replaced CockroachDB, when they found out it has trouble scaling past a point. I really hope they can get their act together in terms of communication and marketing -- it's sadly "the most advanced solution you've never heard of".
- geenat 4y ago> Same birdie told me Aerospike has customers who replaced CockroachDB, when they found out it has trouble scaling past a point. Elaborate?
- notmymainlogin 4y agoNah, I'd rather be vague. ;-) The issue, I'm guessing, was that CockroachDB couldn't maintain the required latency bounds and availability past a certain point for the customer. Those are always where Aerospike wins (latency, availability, while scaling). milliseconds vs microseconds. The limiting factor isn't how many operations the platform can handle, it's how many operations/transactions you can generate before the server will even break a sweat. I'm just a lowly engineer, and while I'll say some stuff on a throwaway account, this is really the marketing department's purview.... Casual googling confirms it's public information that Schwab runs ... nearly everything on it, and it's a much smaller number of servers than you'd think.
- paulryanrogers 4y agoWow, Schwab runs an AGPL database?! Guessing they've a commercial license.
- olddustytrail 4y agoDoesn't it seem equally likely they haven't made any changes to the codebase? Or have simply contributed them to the project?
- paulryanrogers 4y agoAGPL requires making 'full' source of a network service available to network users. Doubtful Schwab would do that.
- olddustytrail 4y ago
- rad_gruchalski 4y agoHaven’t used them in production because their license doesn’t allow operating as a service. I have used an alternative.
- tluyben2 4y agoWhat does that mean? What service do you have that is disallowed? We are evaluating it and are finding the license page confusing. Yugabyte seems friendlier but it seems even less talked about.
- powerhour 4y agoCockroachdb uses the BSL which means you can't host it as a service for others. It's more of a "source available" license (that eventually converts to Apache, after a few years).
- rad_gruchalski 4y agoI used YugabyteDB. License was one thing: database is Apache 2 licensed and no strings attached, you can operate it as a service yourself. Not so with CockroachDB, what powerhour said. Other reasons: it’s Postgres code under the hood, not a rewrite, so you can use existing extensions. I have even contributed the support for FDWs, which I use in production. More here https://gruchalski.com/tags/yugabytedb/ https://gruchalski.com/tags/yugabytedb/ and here https://github.com/radekg/yugabyte-db-multi-tenant-paas-demo https://github.com/radekg/yugabyte-db-multi-tenant-paas-demo.
- tluyben2 4y agoThank you for that feedback!
- sean0- 4y agoTo be clear, you can use CockroachDB if you're a service. What is prohibited, however, is you can't take CockroachDB and try and offer a hosted CockroachDB-as-a-Service to external organizations. But if your organization has a DB-as-a-Service offering internally, you can totally add crdb to the mix (this is pretty common, as a matter of fact). If your org offers a service and its backed by crdb, you can totally do that, too.
- mduggles 4y agoYeah I’d be glad to talk about it. Was part of a team that was asked to design a whistleblowing software platform for large corporate and government customers. The rest of the stack was RDS Postgres but this whistleblowing app had super different requirements. We would need to run in basically every region and also not exclusively in AWS. After trying some different options it became clear that Cockroach was the best option by far. The strongly consistent nature of Cockroach made it a good choice for a SoR like whistleblowing unlike say Mongo. The ease of deployment and the stability of the Raft consensus protocol for managing ranges of data was better than expected. For me having done a global rollout of it, I’d recommend it with very few caveats. You do need to train developers on what queries it can and cannot support, since it does have Postgres client support but obviously cannot handle the full range of queries. You also need to have lots of alerting around the Raft log, since that is your bread and butter in terms of “is this working or not”. However in aggressive testing, removing nodes with no warning under load, simulating pretty severe networking problems, etc, it has held up remarkably well.
- potamic 4y agoWhat's your global setup like, if you may? Running software for government means you generally want your data isolated from everything else. Wondering where cockroach fits in.
- jollybean 4y agoSo why would cockroach be a fit here? That doesn't sound like an app that's going to have to process a lot of data at all. Is it the fact you can get clusters synched really well across regions? And is that really a problem for low transaction apps today?
- deleted 4y ago[deleted]
- tluyben2 4y agoMaybe people who answer/read this also evaluated yugabytedb, which has a much friendlier license (apache 2)? We are evaluating different solutions and we depend on some Postgres compatibility as we have a lot of code to that effect. Yugabyte and cockroach come up all the time.
- punkpeye 4y agoGood find. No experience, but adding my list to try.
- punkpeye 4y agoAll these comments and still no one really talking about their experience... For what it is worth, the biggest put off for me was that they no longer allow to export data in PostgreSQL compatible way, i.e. You are locked in when you start using them.
- sean0- 4y agoWhy can't you `EXPORT`[1] from crdb and then `COPY FROM`[2] into PostgreSQL? [1] https://www.cockroachlabs.com/docs/stable/export.html https://www.cockroachlabs.com/docs/stable/export.html [2] https://www.postgresql.org/docs/current/sql-copy.html https://www.postgresql.org/docs/current/sql-copy.html
- sean0- 4y agoYes. AMA and I’ll do my best to answer.
- deleted 4y ago[deleted]
- Dan_Rivel 4y agoWe tried it for a payment system, the concurrency issues where an absolute nightmare to deal with.Especially in combination with Hibernate. In serializable isolation level some queries would just never finish, no matter how many retries you gave it.
- egorfine 4y agoLaunched it on a t3.nano (0.5GB RAM) in a single node configuration for two extremely small tables - got oom kills in couple of hours. Launched on a t3.micro (1GB RAM) - same. Launched on a t3.small (2GB RAM) - finally it works and doesn't crash. So, min 2GB RAM to serve about 10 (as in: ten, not thousands) records queried every other second.