7 ms·

Show HN: KubeDB – Kubernetes-ready production-grade databases

108 points by tamalsaha001 8y ago

tn890 8y ago
Can someone explain to me why I'd run this over a StatefulSet of whatever software I want to deploy? I've been skimming the docs and can't find a good reason TBH.
- dijit 8y ago
  I think the value add is the various "for free" stuff that you get listed here: https://kubedb.com/docs/0.9.0/concepts/what-is-kubedb/overview/ https://kubedb.com/docs/0.9.0/concepts/what-is-kubedb/overvi...
- striking 8y ago
  The Postgres clustering advertised here[0] seems like a great reason to me. 0: https://kubedb.com/docs/0.9.0/guides/postgres/ https://kubedb.com/docs/0.9.0/guides/postgres/
  - newaccoutnas 8y ago
    How does that compare to something like CitusDB I wonder?
    - sciurus 8y ago
      It looks like a standard single r/w master + multiple r/o slave topology with the addition of their own leader election. I'd want to understand how the latter works and how it is tested very well before I'd trust it. https://kubedb.com/docs/0.9.0/guides/postgres/clustering/ha_cluster/ https://kubedb.com/docs/0.9.0/guides/postgres/clustering/ha_... https://github.com/kubedb/postgres/blob/master/pkg/leader_election/leader_election.go https://github.com/kubedb/postgres/blob/master/pkg/leader_el...
- snug 8y ago
  StatefulSets are not "smart." You would need to manage and configure any kind of replications, leader elections, backups, etc. Operators like KubeDB make this easier to manage, and can make stateful applications more "Cloud Native"
noahdesu 8y ago
This seems to serve a purpose similar to Rook [0] for deploying distributed (or non-distributed) storage systems, including database. For example, Rook can deploy CockroachDB and also Ceph, which would have a lot of the same challenges as deploying some of the databases supported by KubeDB. [0]: https://rook.io https://rook.io disclaimer: red hat employee working some on rook.
- manigandham 8y ago
  Rook stretching to databases looks like a random tangent trying to do everything at once. The project should focus on being a solid distributed storage provider first and let users run databases themselves on those volumes. It's especially strange since these databases already have their own storage replication and would be duplicating work.
kitd 8y ago
My first impression is that your Concepts > Overview section needs to go on your front page. Because it's not obvious just what the product is. Other than that, looks cool.
- weberc2 8y ago
  I thought the front page was quite helpful for figuring out what the product was (once I realized the HN link pointed to the documentation page and not the front page).
  - hosh 8y ago
    Ah! I didn't even notice I was not on the front page!
    - weberc2 8y ago
      Yeah, don’t feel bad. It took me a while as well.
- hosh 8y ago
  Same here. I didn't know what the elevator pitch is. On the other hand, when I looked it up, this sounds like a really interesting project, exposing DBs as application resources. There is something I remember from CloudFoundry (though I don't know if this is working anymore), and that is with adding in dependencies like databases into the specs for an application. I kinda wonder if that could be done: add an annotation on a pod spec which the operator can look for and set up the database (if it doesn't exist) and provision credentials.
dkoston 8y ago
Thanks for putting this together, it's great to have something that covers backups and restore and operation all in one package. I'd recommend updating the name to be more descriptive. Anything xDB sounds like it in itself is a database. That's what I was expecting. KubeDBOperator or similar would let us know: it's related to K8s, it has to do with databases, it likely manages deployments with HA/failover/backups. As other commenters noted as well, you have to click at least 3 times before you understand what the software does based on the above link. Linking to the home page rather than the welcome would be a better choice for those not currently familiar with your project. Either way, appreciate all the work that went into this project!
- atombender 8y ago
  I agree about the confusing naming. They could reverse it: DBKube!
xena 8y ago
On by default analytics makes me not want to touch this, sorry.
- snug 8y ago
  +1 I hope this is just pre-release. I imagine once 1.0 it'll be opt-in.
  - merlinsbrain 8y ago
    I’ve never understood why people are willing to give all their data to Facebook or any other hosted product which collects data, but are unhappy when open source projects collect anonymized metrics to improve the product. In this case there’s a flag advertized to turn it off, hopefully someone will be reading the docs before it’s deployed in production? Do you think AWS and GCP don’t collect metrics? Happy to hear the other side, my knee-jerk reaction is also usually “you don’t need this data”, but when thinking as an owner, i would find this data invaluable to improve the product, getting funding (please donate, there are X millions of users) etc.
- ofrzeta 8y ago
  "To disable stats collection, run the operator with the flag --analytics=false"
- tamalsaha001 8y ago
  I hear you. The goal of our analytics collection is to know the # of active users of our product. So, we send an event to Google Analytics when a command is run or periodic pings for long running (> 1 hr) commands. You can disable it by setting the flag --enable-analytics=false. You can see further discussion on this matter here: https://github.com/cncf/toc/issues/107#issue-311155675 https://github.com/cncf/toc/issues/107#issue-311155675 Lastly, everything is open source. So, you are free to read the code and modify and use your modified version if that's what you prefer. xref: https://github.com/kubedb/operator/blob/0.9.0/pkg/cmds/root.go#L25 https://github.com/kubedb/operator/blob/0.9.0/pkg/cmds/root....
gigatexal 8y ago
Can TLS encryption be used here and does it support encrypting the storage?
- tamalsaha001 8y ago
  Are you talking about TLS encryption for connection url? If so, it is supported for Elasticseach. The others ones are not there yet. Though it might be possible using the available configuration options. We are also looking to modifying the snapshot option to store encrypted snapshots in future releases.
  - gigatexal 8y ago
    That and also mTLS for data in flight.
bogomipz 8y ago
Does anyone know if there are plans for a Cassandra operator? Or is running Cassandra on K8S just not recommended?
- bbromhead 8y ago
  https://github.com/instaclustr/cassandra-operator https://github.com/instaclustr/cassandra-operator :)
  - bogomipz 8y ago
    Thanks for the link, do you have experience or feedback on using this?
wstrange 8y ago
Really nice work! Question: How are snapshots implemented? Are you using k8s native snapshots (which I think are alpha?) or other? EDIT: Looks like snapshots are per db feature: https://github.com/kubedb/project/issues/337 https://github.com/kubedb/project/issues/337 Hopefully Kubedb can leverage K8S snapshots when they mature
- tamalsaha001 8y ago
  Thank you! We are using DB specific tools for taking snapshot. Example, mysqldump for MySQL etc. We are looking into k8s native snapshots for future releases. Here is the tracking issue: https://github.com/kubedb/project/issues/337 https://github.com/kubedb/project/issues/337
lbj 8y ago
Is it just me or is the number of clicks required to get a feel for the project ridiculously high? I gave up
gigatexal 8y ago
No MySQL clustering means no adoption from me for now.
- tamalsaha001 8y ago
  I understand. This is one of primary feature we targeting to suppose in the next quarter. Can you tell me what type of clustering support do you use? Example: Percona, Galera cluster, Vitess ?
  - gigatexal 8y ago
    Hah if you can make vitess not a terrible experience that would be awesome. But just Percona clustering would do.
merb 8y ago
are there any docs what you can set for the crd values? I only see some examples but I do not see any full crd api doc. something like: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.13/#daemonset-v1-apps https://kubernetes.io/docs/reference/generated/kubernetes-ap...
alexnewman 8y ago
Is this production worthy?
tie_ 8y ago
Kudos for the interesting approach! I do think that DB management on Kubernetes belongs to operators, and maintaining a collection of them would be great. This being said, you should not be using TPR, but rather CRDs (see https://coreos.com/blog/custom-resource-kubernetes-v17 https://coreos.com/blog/custom-resource-kubernetes-v17). The welcome video is not really convincing either - condensing it to an animated gif would make it much more attractive imho.

108 points by tamalsaha001 8y ago

Can someone explain to me why I'd run this over a StatefulSet of whatever software I want to deploy? I've been skimming the docs and can't find a good reason TBH.

dijit 8y ago
I think the value add is the various "for free" stuff that you get listed here: https://kubedb.com/docs/0.9.0/concepts/what-is-kubedb/overview/ https://kubedb.com/docs/0.9.0/concepts/what-is-kubedb/overvi...
striking 8y ago
The Postgres clustering advertised here[0] seems like a great reason to me. 0: https://kubedb.com/docs/0.9.0/guides/postgres/ https://kubedb.com/docs/0.9.0/guides/postgres/
- newaccoutnas 8y ago
  How does that compare to something like CitusDB I wonder?
  - sciurus 8y ago
    It looks like a standard single r/w master + multiple r/o slave topology with the addition of their own leader election. I'd want to understand how the latter works and how it is tested very well before I'd trust it. https://kubedb.com/docs/0.9.0/guides/postgres/clustering/ha_cluster/ https://kubedb.com/docs/0.9.0/guides/postgres/clustering/ha_... https://github.com/kubedb/postgres/blob/master/pkg/leader_election/leader_election.go https://github.com/kubedb/postgres/blob/master/pkg/leader_el...
snug 8y ago
StatefulSets are not "smart." You would need to manage and configure any kind of replications, leader elections, backups, etc. Operators like KubeDB make this easier to manage, and can make stateful applications more "Cloud Native"

noahdesu 8y ago

This seems to serve a purpose similar to Rook [0] for deploying distributed (or non-distributed) storage systems, including database. For example, Rook can deploy CockroachDB and also Ceph, which would have a lot of the same challenges as deploying some of the databases supported by KubeDB. [0]: https://rook.io https://rook.io disclaimer: red hat employee working some on rook.

manigandham 8y ago
Rook stretching to databases looks like a random tangent trying to do everything at once. The project should focus on being a solid distributed storage provider first and let users run databases themselves on those volumes. It's especially strange since these databases already have their own storage replication and would be duplicating work.

kitd 8y ago

My first impression is that your Concepts > Overview section needs to go on your front page. Because it's not obvious just what the product is. Other than that, looks cool.

weberc2 8y ago
I thought the front page was quite helpful for figuring out what the product was (once I realized the HN link pointed to the documentation page and not the front page).
- hosh 8y ago
  Ah! I didn't even notice I was not on the front page!
  - weberc2 8y ago
    Yeah, don’t feel bad. It took me a while as well.
hosh 8y ago
Same here. I didn't know what the elevator pitch is. On the other hand, when I looked it up, this sounds like a really interesting project, exposing DBs as application resources. There is something I remember from CloudFoundry (though I don't know if this is working anymore), and that is with adding in dependencies like databases into the specs for an application. I kinda wonder if that could be done: add an annotation on a pod spec which the operator can look for and set up the database (if it doesn't exist) and provision credentials.

dkoston 8y ago

Thanks for putting this together, it's great to have something that covers backups and restore and operation all in one package. I'd recommend updating the name to be more descriptive. Anything xDB sounds like it in itself is a database. That's what I was expecting. KubeDBOperator or similar would let us know: it's related to K8s, it has to do with databases, it likely manages deployments with HA/failover/backups. As other commenters noted as well, you have to click at least 3 times before you understand what the software does based on the above link. Linking to the home page rather than the welcome would be a better choice for those not currently familiar with your project. Either way, appreciate all the work that went into this project!

atombender 8y ago
I agree about the confusing naming. They could reverse it: DBKube!

xena 8y ago

On by default analytics makes me not want to touch this, sorry.

snug 8y ago
+1 I hope this is just pre-release. I imagine once 1.0 it'll be opt-in.
- merlinsbrain 8y ago
  I’ve never understood why people are willing to give all their data to Facebook or any other hosted product which collects data, but are unhappy when open source projects collect anonymized metrics to improve the product. In this case there’s a flag advertized to turn it off, hopefully someone will be reading the docs before it’s deployed in production? Do you think AWS and GCP don’t collect metrics? Happy to hear the other side, my knee-jerk reaction is also usually “you don’t need this data”, but when thinking as an owner, i would find this data invaluable to improve the product, getting funding (please donate, there are X millions of users) etc.
ofrzeta 8y ago
"To disable stats collection, run the operator with the flag --analytics=false"
tamalsaha001 8y ago
I hear you. The goal of our analytics collection is to know the # of active users of our product. So, we send an event to Google Analytics when a command is run or periodic pings for long running (> 1 hr) commands. You can disable it by setting the flag --enable-analytics=false. You can see further discussion on this matter here: https://github.com/cncf/toc/issues/107#issue-311155675 https://github.com/cncf/toc/issues/107#issue-311155675 Lastly, everything is open source. So, you are free to read the code and modify and use your modified version if that's what you prefer. xref: https://github.com/kubedb/operator/blob/0.9.0/pkg/cmds/root.go#L25 https://github.com/kubedb/operator/blob/0.9.0/pkg/cmds/root....

gigatexal 8y ago

Can TLS encryption be used here and does it support encrypting the storage?

tamalsaha001 8y ago
Are you talking about TLS encryption for connection url? If so, it is supported for Elasticseach. The others ones are not there yet. Though it might be possible using the available configuration options. We are also looking to modifying the snapshot option to store encrypted snapshots in future releases.
- gigatexal 8y ago
  That and also mTLS for data in flight.

bogomipz 8y ago

Does anyone know if there are plans for a Cassandra operator? Or is running Cassandra on K8S just not recommended?

bbromhead 8y ago
https://github.com/instaclustr/cassandra-operator https://github.com/instaclustr/cassandra-operator :)
- bogomipz 8y ago
  Thanks for the link, do you have experience or feedback on using this?

wstrange 8y ago

Really nice work! Question: How are snapshots implemented? Are you using k8s native snapshots (which I think are alpha?) or other? EDIT: Looks like snapshots are per db feature: https://github.com/kubedb/project/issues/337 https://github.com/kubedb/project/issues/337 Hopefully Kubedb can leverage K8S snapshots when they mature

tamalsaha001 8y ago
Thank you! We are using DB specific tools for taking snapshot. Example, mysqldump for MySQL etc. We are looking into k8s native snapshots for future releases. Here is the tracking issue: https://github.com/kubedb/project/issues/337 https://github.com/kubedb/project/issues/337

lbj 8y ago

Is it just me or is the number of clicks required to get a feel for the project ridiculously high? I gave up

gigatexal 8y ago

No MySQL clustering means no adoption from me for now.

tamalsaha001 8y ago
I understand. This is one of primary feature we targeting to suppose in the next quarter. Can you tell me what type of clustering support do you use? Example: Percona, Galera cluster, Vitess ?
- gigatexal 8y ago
  Hah if you can make vitess not a terrible experience that would be awesome. But just Percona clustering would do.

merb 8y ago

are there any docs what you can set for the crd values? I only see some examples but I do not see any full crd api doc. something like: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.13/#daemonset-v1-apps https://kubernetes.io/docs/reference/generated/kubernetes-ap...

alexnewman 8y ago

Is this production worthy?

tie_ 8y ago

Kudos for the interesting approach! I do think that DB management on Kubernetes belongs to operators, and maintaining a collection of them would be great. This being said, you should not be using TPR, but rather CRDs (see https://coreos.com/blog/custom-resource-kubernetes-v17 https://coreos.com/blog/custom-resource-kubernetes-v17). The welcome video is not really convincing either - condensing it to an animated gif would make it much more attractive imho.