3 ms·

I used to think of stateful applications on top of k8s as an anti-pattern until I learned a bit more about some databases uses and changed my entire opinion abo

by Bayart 4y ago

I used to think of stateful applications on top of k8s as an anti-pattern until I learned a bit more about some databases uses and changed my entire opinion about what k8s is. If you're coming from the container world, it makes sense to think of orchestration as a way to scale stateless services.

cogman10 4y ago
I'd be glad to hear those usecases. I'm still in the camp of thinking it's not a great idea. I have a hard time thinking coordination between something like postgres and the volume wouldn't be an absolute nightmare to handle. What sorts of volume drivers do you use there? How do you get over the network latency? How does scaling end up working?
- Bayart 4y ago
  I've not used postgres-on-k8s in any real production workload so my opinion on the matter isn't worth glozing about. I'd rather recommend the blog posts and docs fly.io did on it[1]. They themselves rely on Stolon[2]. AFAIK the secret sauce is postgres' ability for streaming replication[3], ie. pass data across instances as it's ingressed. Add some glue and you've got a cluster. As far as storage goes, k8s supports topology awareness[4] and enables collocating pods and PVs. There are CSI drivers for most file systems you'd encounter in the enterprise[5][6]. > How do you get over the network latency? Ultimately there are incompressible limits you'll hit (CAP theorem and all that). So you do what everybody does in computing : you cheat by exploiting human perception and treating your reads and writes differently[7]. I'm by no means an expert on the subject, but I hope I've answered some of your questions. --- [1]: https://fly.io/blog/globally-distributed-postgres/ https://fly.io/blog/globally-distributed-postgres/ [2]: https://github.com/sorintlab/stolon https://github.com/sorintlab/stolon [3]: https://wiki.postgresql.org/wiki/Streaming_Replication https://wiki.postgresql.org/wiki/Streaming_Replication [4]: https://kubernetes.io/blog/2018/10/11/topology-aware-volume-provisioning-in-kubernetes/ https://kubernetes.io/blog/2018/10/11/topology-aware-volume-... [5]: https://kubernetes.io/blog/2019/01/15/container-storage-interface-ga/ https://kubernetes.io/blog/2019/01/15/container-storage-inte... [6]: https://kubernetes-csi.github.io/docs/drivers.html https://kubernetes-csi.github.io/docs/drivers.html [7]: https://fly.io/blog/globally-distributed-postgres/ https://fly.io/blog/globally-distributed-postgres/
- vbezhenar 4y ago
  You can run database on local volume and not deal with network at all if you don't like it. You can dedicate node for your postrgres instance. Or you can start simple until you hit performance wall and then gradually implement those things. Kubernetes is pretty flexible in that regard.

by Bayart 4y ago

cogman10 4y ago

I'd be glad to hear those usecases. I'm still in the camp of thinking it's not a great idea. I have a hard time thinking coordination between something like postgres and the volume wouldn't be an absolute nightmare to handle. What sorts of volume drivers do you use there? How do you get over the network latency? How does scaling end up working?

Bayart 4y ago
I've not used postgres-on-k8s in any real production workload so my opinion on the matter isn't worth glozing about. I'd rather recommend the blog posts and docs fly.io did on it[1]. They themselves rely on Stolon[2]. AFAIK the secret sauce is postgres' ability for streaming replication[3], ie. pass data across instances as it's ingressed. Add some glue and you've got a cluster. As far as storage goes, k8s supports topology awareness[4] and enables collocating pods and PVs. There are CSI drivers for most file systems you'd encounter in the enterprise[5][6]. > How do you get over the network latency? Ultimately there are incompressible limits you'll hit (CAP theorem and all that). So you do what everybody does in computing : you cheat by exploiting human perception and treating your reads and writes differently[7]. I'm by no means an expert on the subject, but I hope I've answered some of your questions. --- [1]: https://fly.io/blog/globally-distributed-postgres/ https://fly.io/blog/globally-distributed-postgres/ [2]: https://github.com/sorintlab/stolon https://github.com/sorintlab/stolon [3]: https://wiki.postgresql.org/wiki/Streaming_Replication https://wiki.postgresql.org/wiki/Streaming_Replication [4]: https://kubernetes.io/blog/2018/10/11/topology-aware-volume-provisioning-in-kubernetes/ https://kubernetes.io/blog/2018/10/11/topology-aware-volume-... [5]: https://kubernetes.io/blog/2019/01/15/container-storage-interface-ga/ https://kubernetes.io/blog/2019/01/15/container-storage-inte... [6]: https://kubernetes-csi.github.io/docs/drivers.html https://kubernetes-csi.github.io/docs/drivers.html [7]: https://fly.io/blog/globally-distributed-postgres/ https://fly.io/blog/globally-distributed-postgres/
vbezhenar 4y ago
You can run database on local volume and not deal with network at all if you don't like it. You can dedicate node for your postrgres instance. Or you can start simple until you hit performance wall and then gradually implement those things. Kubernetes is pretty flexible in that regard.