5 ms·

CockroachDB 20.1

127 points by dilloc 6y ago

tyingq 6y ago
Looking at the product comparison chart[1], there's quite a lot of features that aren't in the open source "core" CockroachDB. That's fine of course, but the one that seems concerning is the backup/restore. Is there a reasonable and reliable way to do backups and restores with just the "core" open source product? [1] https://www.cockroachlabs.com/compare/ https://www.cockroachlabs.com/compare/
- shawabawa3 6y ago
  I may be misremembering but I'm pretty sure you can backup like any postgres database with e.g. pg_dump I think the premium distributed backup/restore thing is for backing up separate regional clusters individually edit: looks like the premium option is a nice "BACKUP" command that handles uploading or downloading from cloud storage (e.g. s3) automatically. but for free you get "cockroach dump" which is similar to pg_dump
  - latch 6y ago
    The enterprise license gets you incremental backups. cockroach dump is similar to pg_dump, but it's worth pointing out that pg offers a lot more than just pg_dump. Things like pg_basebackup (and accompanying tools) and various replication strategies and capabilities (e.g. recovery_min_apply_delay) make pg a vastly safer option from a DR point of view.
- pritambaral 6y ago
  Is that still open source? Didn't they change to the BSL?
- latch 6y ago
  No, there isn't. The only option is the dump command (full backup). But it's slow. And the restore is unreasonably slow. If you have a medium-sized database, you'll likely have to accept uncomfortably large (possibly even disastrous) data-loss window and a long downtime while you restore. There's even cases where it can't work (1). The free options from PG (namely things like barman and the built-in replication) is far superior to what the paid version of CockroachDB offers, let alone the community edition. I wish they'd let you take a rocksdb checkpoint of a single node and restore that into another cluster. This should work if your replica == node count (a common setup). Getting access to the checkpoint isn't complicated, but recovering this with the leaseholders and cluster config baked into the database requires more insight into their abstractions than I have. Feels like something they need anyways, because, as-is, a permanent loss of 2 nodes is impossible to recover from (2) (1) https://github.com/cockroachdb/cockroach/issues/28948 https://github.com/cockroachdb/cockroach/issues/28948 (2) https://github.com/cockroachdb/cockroach/issues/17186 https://github.com/cockroachdb/cockroach/issues/17186
- sandstrom 6y ago
  Yeah, backup is missing. This question has been asked in the comments of every HN post about a new release. It doesn’t make sense, they are missing out on lots of enthusiast/hacker adoption, people that can’t afford enterprise anyway. Having backup as a “differentiating feature” for enterprise is such a stupid idea.
  - tyingq 6y ago
    Well, and eventually sets you up for some bad press. I assume a "I chose CRDB, and now I've lost all my data" story will hit at some point. And it won't be obvious for all the readers that they weren't a paying customer.
ithkuil 6y ago
I see an untrusted cert: "Issued by: allot.com/emailAddress=info@allot.com". Am I the only one? EDIT: https://www.allot.com/service-providers/url-traffic-filtering/; https://www.allot.com/service-providers/url-traffic-filterin... still not sure if it's on my side, or if the CDN (netlify in this case) is applying this to some regions (I'm in EU)
- kkirsche 6y ago
  Yes, I see a trusted let’s encrypt cert
- tyingq 6y ago
  Looks okay on ssllabs: https://www.ssllabs.com/ssltest/analyze.html?d=www.cockroachlabs.com&latest https://www.ssllabs.com/ssltest/analyze.html?d=www.cockroach...
- corobo 6y ago
  Sounds like something's intercepting your connection. I would imagine, but don't necessarily recommend, if you allowed the connection anyway you'd get a blocked page from your firewall service
- cpach 6y ago
  I don’t see that. If your on your work network, it could be your employer’s security system that is intercepting your connection. A possible explanation could be that the domain cockroachlabs.com is not yet categorized in the security system.
- tyingq 6y ago
  "allot.com" appears to be one of those "MITM Corporate/Parental Nanny Software" companies. You're either being MITMed by your company or your parents :)
  - ithkuil 6y ago
    Why just this site?
gigatexal 6y ago
Online primary key migrations is some interesting voodoo. We ran it in production from the v2.x days before they moved to a new versioning schema and the support was really good. The paid support is worth the enterprise license. It's also good to see the added work on supporting ORMs for shops that use it. We used SQLAlchemy heavily and had to do a lot of work to fix our app to work around the transaction retry errors which CRDB has been working on lessening in recent releases.
- rsanders 6y ago
  Retrying transactions is something every app ought to handle, but it's rare enough that most codebases I've seen just punt on it.
  - gigatexal 6y ago
    The thing is before CRDB the application was operating just fine in MySQL's default isolation level.
stuff4ben 6y ago
Maybe I've been living under a rock, but this is the first I've heard of the Business Source License (BSL). https://www.cockroachlabs.com/docs/stable/licensing-faqs.html https://www.cockroachlabs.com/docs/stable/licensing-faqs.htm... So after it converts to Apache 2.0 licensing, how does that affect redistribution of CockroachDB if I include it in my commercial applications?
- lukevp 6y ago
  First for me too. I like the idea of a time limited conversion to Apache. Companies w easily AWS’d apps need some leverage and having older versions being the only open source ones makes sense. I would prefer a non commercial source available license as well or open core but this doesn’t seem too unreasonable.

127 points by dilloc 6y ago

Looking at the product comparison chart[1], there's quite a lot of features that aren't in the open source "core" CockroachDB. That's fine of course, but the one that seems concerning is the backup/restore. Is there a reasonable and reliable way to do backups and restores with just the "core" open source product? [1] https://www.cockroachlabs.com/compare/ https://www.cockroachlabs.com/compare/

shawabawa3 6y ago
I may be misremembering but I'm pretty sure you can backup like any postgres database with e.g. pg_dump I think the premium distributed backup/restore thing is for backing up separate regional clusters individually edit: looks like the premium option is a nice "BACKUP" command that handles uploading or downloading from cloud storage (e.g. s3) automatically. but for free you get "cockroach dump" which is similar to pg_dump
- latch 6y ago
  The enterprise license gets you incremental backups. cockroach dump is similar to pg_dump, but it's worth pointing out that pg offers a lot more than just pg_dump. Things like pg_basebackup (and accompanying tools) and various replication strategies and capabilities (e.g. recovery_min_apply_delay) make pg a vastly safer option from a DR point of view.
pritambaral 6y ago
Is that still open source? Didn't they change to the BSL?
latch 6y ago
No, there isn't. The only option is the dump command (full backup). But it's slow. And the restore is unreasonably slow. If you have a medium-sized database, you'll likely have to accept uncomfortably large (possibly even disastrous) data-loss window and a long downtime while you restore. There's even cases where it can't work (1). The free options from PG (namely things like barman and the built-in replication) is far superior to what the paid version of CockroachDB offers, let alone the community edition. I wish they'd let you take a rocksdb checkpoint of a single node and restore that into another cluster. This should work if your replica == node count (a common setup). Getting access to the checkpoint isn't complicated, but recovering this with the leaseholders and cluster config baked into the database requires more insight into their abstractions than I have. Feels like something they need anyways, because, as-is, a permanent loss of 2 nodes is impossible to recover from (2) (1) https://github.com/cockroachdb/cockroach/issues/28948 https://github.com/cockroachdb/cockroach/issues/28948 (2) https://github.com/cockroachdb/cockroach/issues/17186 https://github.com/cockroachdb/cockroach/issues/17186
sandstrom 6y ago
Yeah, backup is missing. This question has been asked in the comments of every HN post about a new release. It doesn’t make sense, they are missing out on lots of enthusiast/hacker adoption, people that can’t afford enterprise anyway. Having backup as a “differentiating feature” for enterprise is such a stupid idea.
- tyingq 6y ago
  Well, and eventually sets you up for some bad press. I assume a "I chose CRDB, and now I've lost all my data" story will hit at some point. And it won't be obvious for all the readers that they weren't a paying customer.

ithkuil 6y ago

I see an untrusted cert: "Issued by: allot.com/emailAddress=info@allot.com". Am I the only one? EDIT: https://www.allot.com/service-providers/url-traffic-filtering/; https://www.allot.com/service-providers/url-traffic-filterin... still not sure if it's on my side, or if the CDN (netlify in this case) is applying this to some regions (I'm in EU)

kkirsche 6y ago
Yes, I see a trusted let’s encrypt cert
tyingq 6y ago
Looks okay on ssllabs: https://www.ssllabs.com/ssltest/analyze.html?d=www.cockroachlabs.com&latest https://www.ssllabs.com/ssltest/analyze.html?d=www.cockroach...
corobo 6y ago
Sounds like something's intercepting your connection. I would imagine, but don't necessarily recommend, if you allowed the connection anyway you'd get a blocked page from your firewall service
cpach 6y ago
I don’t see that. If your on your work network, it could be your employer’s security system that is intercepting your connection. A possible explanation could be that the domain cockroachlabs.com is not yet categorized in the security system.
tyingq 6y ago
"allot.com" appears to be one of those "MITM Corporate/Parental Nanny Software" companies. You're either being MITMed by your company or your parents :)
- ithkuil 6y ago
  Why just this site?

gigatexal 6y ago

Online primary key migrations is some interesting voodoo. We ran it in production from the v2.x days before they moved to a new versioning schema and the support was really good. The paid support is worth the enterprise license. It's also good to see the added work on supporting ORMs for shops that use it. We used SQLAlchemy heavily and had to do a lot of work to fix our app to work around the transaction retry errors which CRDB has been working on lessening in recent releases.

rsanders 6y ago
Retrying transactions is something every app ought to handle, but it's rare enough that most codebases I've seen just punt on it.
- gigatexal 6y ago
  The thing is before CRDB the application was operating just fine in MySQL's default isolation level.

stuff4ben 6y ago

Maybe I've been living under a rock, but this is the first I've heard of the Business Source License (BSL). https://www.cockroachlabs.com/docs/stable/licensing-faqs.html https://www.cockroachlabs.com/docs/stable/licensing-faqs.htm... So after it converts to Apache 2.0 licensing, how does that affect redistribution of CockroachDB if I include it in my commercial applications?

lukevp 6y ago
First for me too. I like the idea of a time limited conversion to Apache. Companies w easily AWS’d apps need some leverage and having older versions being the only open source ones makes sense. I would prefer a non commercial source available license as well or open core but this doesn’t seem too unreasonable.