4 ms·
It's a bit scary when you're using K8S ephemeral storage and depending on a basebackup from another pod, but you can also set it up to restore from your usual b
by abkfenris 9y ago
It's a bit scary when you're using K8S ephemeral storage and depending on a basebackup from another pod, but you can also set it up to restore from your usual backup system (WAL-E, pghoard, barman, pgBackrest). That way any time you lose a pod, you get a test of your backup system on restore.
- eicnix 9y agoThere is currently some work being done for improving local storage usage in k8s. https://github.com/kubernetes/community/pull/306 https://github.com/kubernetes/community/pull/306 Additionally k8s can be configured to restart pods on the same node so you can use the local node storage for persistence and rely on your DB for replication.
- icebraining 9y agoyou can also set it up to restore from your usual backup system (WAL-E, pghoard, barman, pgBackrest) That sounds interesting, has anyone documented that kind of setup?
- abkfenris 9y agoI'm working on one. I am waiting for a PR to land and a release to get generated for a bug that I found. If the master pod in a StatefulSet is deleted, and kubernetes sets up a new one too fast, it currently keeps the master lock and failover never happens (it also tries to bootstrap from itself). Once those land I'll work on a writeup of that setup I've got a few Postgres instances running as single node Deployments and StatefulSets right now that backup and restore using pghoard for simple services that I've documented here http://alexkerney.com/2016/10/pghoard-kubernetes.html http://alexkerney.com/2016/10/pghoard-kubernetes.html I do need to add a note about using livenessprobes to make sure the restore completed successfully. Sometimes these pods are cycled every day as half my K8S cluster is preemptable.