4 ms·
Hey thanks for the article. I know etcd upgrades can look complex but upgrading distributed databases live is always going to be quite non-trivial. That said f
by philips 8y ago
Hey thanks for the article. I know etcd upgrades can look complex but upgrading distributed databases live is always going to be quite non-trivial.
That said for many people taking some downtime on their Kube API server isn't the end of the world. The system, by design, can work OK for sometime in a degraded state: workloads keep running.
A few things that I do want to try to clarify:
1) The strict documented upgrade path for etcd is because testing matrixes just get too complicated. There aren't really technical limitations as much as wanting to ensure recommendations are made based on things that have been tested. The documentation is all here: https://github.com/coreos/etcd/tree/master/Documentation/upgrades https://github.com/coreos/etcd/tree/master/Documentation/upg...
2) Live etcd v2 API -> etcd v3 API migration for Kubernetes was never a priority for the etcd team at CoreOS because we never shipped a supported product that used Kube + etcd v2. Several community members volunteered to make it better but it never really came together. We feel bad about the mess but it is a consequence of not having that itch to scratch as they say.
3) Several contributors, notably Joe Betz of Google, have been working to keep older minor versions of etcd patched. For example 3.1.17 was released 30 days ago and the first release of 3.1 was 1.5 years ago. These longer lived branches intend to be bug fix only.
- kevin_nisbet 8y agoAwesome, thanks for the clarifying items. When going through this, I did test it and see that the upgrade appeared to work, until I think it was 3.3 where it would panic, but didn't want to rely on the undefined / untested behaviour, even if it seemed to work in a lab. The interview was long, so as we were cutting it down I think this aspect got lost. And thanks for the hard work on etcd.
- geertj 8y ago> upgrading distributed databases live is always going to be quite non-trivial. I understand that the implementation of live upgrades for a distributed database will be complex but this post is about the user experience. Given enough resources, is there a reason that it can't be a single "upgrade now" command? Or maybe slightly more real-world, a 3 step process like: "stage update" -> "test update" -> "start update".
- piva00 8y agoI agree with you, tooling is still lacking a lot in UX on k8s world.
- philips 8y agoThat is the process: https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md#upgrade-procedure https://github.com/coreos/etcd/blob/master/Documentation/upg... for i in members Replace etcd binary Restart etcd process