4 ms·

Thanks for the links. Do you know about a project that Targets PostgreSQL as a operator, like etcd?

alexk 10y ago
This one: https://github.com/sorintlab/stolon https://github.com/sorintlab/stolon It is not called the operator, but in many ways it is similar to what Etcd Operator does
- crb 10y ago
  If your pattern requires you to connect to etcd, then you're not taking the best advantage of Kubernetes. Yes, you can connect to an etcd instance outside of Kubernetes, or deploy etcd on Kubernetes. CoreOS developed the etcd operator for the purposes of self-hosting the system KV store, but you can of course run a "user" etcd. A better pattern is relying on the objects provided by Kubernetes, which are themselves backed by a lock service (which happens to be etcd, but that's an implementation detail you shouldn't worry about.) You could, for example, use an Annotation on an object to do leader election. Look at how HA masters are implemented on Kubernetes, for example: https://github.com/kubernetes/kubernetes/blob/master/pkg/client/leaderelection/leaderelection.go https://github.com/kubernetes/kubernetes/blob/master/pkg/cli...
  - achanda358 10y ago
    That's a great point. But doesn't this make it mandatory for the applications to use Go? It would have been more useful if there was a REST interface for leader election.
    - crb 10y ago
      Not at all - it's just a language for which there is a Kubernetes client. The API is 100% REST. We have a Python library underway also: https://github.com/kubernetes-incubator/client-python https://github.com/kubernetes-incubator/client-python
      - achanda358 10y ago
        Right, but I couldn't find a REST API for leader election. If my app is in Go, I can use the leaderelection package here https://github.com/kubernetes/kubernetes/blob/master/pkg/client/leaderelection/leaderelection.go https://github.com/kubernetes/kubernetes/blob/master/pkg/cli.... But if my app is in Python, I won't be able to use that package. In that case I'd need to use etcd or zookeeper on top of k8s. Am I missing something?
        crb 10y ago
        You can reimplement that logic in Python. There's nothing magic about it: you can see in https://github.com/kubernetes/kubernetes/blob/master/pkg/client/leaderelection/resourcelock/endpointslock.go https://github.com/kubernetes/kubernetes/blob/master/pkg/cli... that it just uses a Kubernetes annotation. There's some more information (and a sidecar pod prebaked) at http://blog.kubernetes.io/2016/01/simple-leader-election-with-Kubernetes.html http://blog.kubernetes.io/2016/01/simple-leader-election-wit...
        CyberDem0n 10y ago
        Yes, one can reimplement it in Python, but don't forget: // This implementation does not guarantee that only one client is acting as a // leader (a.k.a. fencing). A client observes timestamps captured locally to // infer the state of the leader election. Thus the implementation is tolerant // to arbitrary clock skew, but is not tolerant to arbitrary clock skew rate.
  - alexk 10y ago
    makes sense, this was written before TPR though, so there is a room for improvement.
sarnowski 10y ago
Yes, we are currently developing a PostgreSQL operator for Kubernetes. We are heavy PostgreSQL users. Spilo[0] supports all kinds of slave configurations, backups, monitoring tools[1] and all advanced capabilities. [0] https://github.com/zalando/spilo https://github.com/zalando/spilo [1] https://github.com/zalando/PGObserver https://github.com/zalando/PGObserver Edit: https://github.com/zalando/patroni https://github.com/zalando/patroni is a better link
- foxylion 10y ago
  Thank you! And Zalando as a bit company behind it is even better. Are you going to use this as is in production, or do you have an internal version it?
  - sarnowski 10y ago
    We don't have a separate internal version. We are already running some hundreds PostgreSQL clusters in AWS + etcd with Spilo/Patroni. The Kubernetes variant is not used yet in production but we plan to migrate the first datasets soonish (probably some weeks).
    - foxylion 10y ago
      This is great to hear, will definitely try it.

alexk 10y ago

This one: https://github.com/sorintlab/stolon https://github.com/sorintlab/stolon It is not called the operator, but in many ways it is similar to what Etcd Operator does

crb 10y ago
If your pattern requires you to connect to etcd, then you're not taking the best advantage of Kubernetes. Yes, you can connect to an etcd instance outside of Kubernetes, or deploy etcd on Kubernetes. CoreOS developed the etcd operator for the purposes of self-hosting the system KV store, but you can of course run a "user" etcd. A better pattern is relying on the objects provided by Kubernetes, which are themselves backed by a lock service (which happens to be etcd, but that's an implementation detail you shouldn't worry about.) You could, for example, use an Annotation on an object to do leader election. Look at how HA masters are implemented on Kubernetes, for example: https://github.com/kubernetes/kubernetes/blob/master/pkg/client/leaderelection/leaderelection.go https://github.com/kubernetes/kubernetes/blob/master/pkg/cli...
- achanda358 10y ago
  That's a great point. But doesn't this make it mandatory for the applications to use Go? It would have been more useful if there was a REST interface for leader election.
  - crb 10y ago
    Not at all - it's just a language for which there is a Kubernetes client. The API is 100% REST. We have a Python library underway also: https://github.com/kubernetes-incubator/client-python https://github.com/kubernetes-incubator/client-python
    - achanda358 10y ago
      Right, but I couldn't find a REST API for leader election. If my app is in Go, I can use the leaderelection package here https://github.com/kubernetes/kubernetes/blob/master/pkg/client/leaderelection/leaderelection.go https://github.com/kubernetes/kubernetes/blob/master/pkg/cli.... But if my app is in Python, I won't be able to use that package. In that case I'd need to use etcd or zookeeper on top of k8s. Am I missing something?
      - crb 10y ago
        You can reimplement that logic in Python. There's nothing magic about it: you can see in https://github.com/kubernetes/kubernetes/blob/master/pkg/client/leaderelection/resourcelock/endpointslock.go https://github.com/kubernetes/kubernetes/blob/master/pkg/cli... that it just uses a Kubernetes annotation. There's some more information (and a sidecar pod prebaked) at http://blog.kubernetes.io/2016/01/simple-leader-election-with-Kubernetes.html http://blog.kubernetes.io/2016/01/simple-leader-election-wit...
        CyberDem0n 10y ago
        Yes, one can reimplement it in Python, but don't forget: // This implementation does not guarantee that only one client is acting as a // leader (a.k.a. fencing). A client observes timestamps captured locally to // infer the state of the leader election. Thus the implementation is tolerant // to arbitrary clock skew, but is not tolerant to arbitrary clock skew rate.
- alexk 10y ago
  makes sense, this was written before TPR though, so there is a room for improvement.

sarnowski 10y ago

Yes, we are currently developing a PostgreSQL operator for Kubernetes. We are heavy PostgreSQL users. Spilo[0] supports all kinds of slave configurations, backups, monitoring tools[1] and all advanced capabilities. [0] https://github.com/zalando/spilo https://github.com/zalando/spilo [1] https://github.com/zalando/PGObserver https://github.com/zalando/PGObserver Edit: https://github.com/zalando/patroni https://github.com/zalando/patroni is a better link

foxylion 10y ago
Thank you! And Zalando as a bit company behind it is even better. Are you going to use this as is in production, or do you have an internal version it?
- sarnowski 10y ago
  We don't have a separate internal version. We are already running some hundreds PostgreSQL clusters in AWS + etcd with Spilo/Patroni. The Kubernetes variant is not used yet in production but we plan to migrate the first datasets soonish (probably some weeks).
  - foxylion 10y ago
    This is great to hear, will definitely try it.