3 ms·

what if you can't use hosted k8s. what if you are on vmware 3 node (no vsphere). what if you at least need consul/etcd, postgresql, haproxy + two nodes (should

by merb 8y ago

what if you can't use hosted k8s. what if you are on vmware 3 node (no vsphere). what if you at least need consul/etcd, postgresql, haproxy + two nodes (should be as highly available as possible). how do you keep everything up to date, how do you deploy (no downtime) without a complex ansible config. for myself I use k8s either bootkubed or kubeadm, I create ignition configs via cloud-config (coreos) and create coreos nodes via the vmware gui. after that I ssh once into either one node (bootkube) or in all nodes and call kubeadm or bootkube after that I have a working cluster. Maintance is done via coreos update operator. and I can upgrade k8s via either kubeadm or via kubectl on bootkube. (so basically the most complex thing on k8s is updating k8s, everything else isn't as hard as people thing. p.s. I do not have a bach/major in tech).

pm90 8y ago
What did you find complex about updating k8s? Serious question since I work closely with k8s and they have a pretty fast release cycle.
- merb 8y ago
  running into bugs and that I need to ssh into the nodes ;) (serious answer) I can basically automate nearly everything, os updates (coreos), deployment updates (ci/cd), however updating etcd or k8s is a manual operation on bare metal, either I'm on bootkube than I need to login and update kubectl, than I need to update all daemonsets/deployments in kube-system (it's not always easy and bootkube has strange bugs, https://github.com/kubernetes-incubator/bootkube/issues/977 https://github.com/kubernetes-incubator/bootkube/issues/977) So one should actually use kubeadm, once again I still need to update kubelet, so I need to ssh into the machine (anyway) I also need to update kubeadm and run it, again a manual operation. *ssh means I need to do it on my own, for small clusters this can be done manually for bigger clusters it can be scripted via ansible or even better create an own software that does it. My point is, that's the only thing which is not "just works" on k8s (yet). Edit: I also run into a bug that I could not upgrade k8s with kubeadm (but thats gonna be fixed soon: https://github.com/kubernetes/kubeadm/issues/727 https://github.com/kubernetes/kubeadm/issues/727) Edit2: the good thing is that reloading kubelet won't kill anything.
  - barrkel 8y ago
    I'd look at using a regular configuration management system for maintaining k8s itself, e.g. Puppet.
- vetrom 8y ago
  Similar to a couple of the other replies, updating kubelet or the like tends to touch enough dependencies that it starts looking sensible to do updates as rolling VM re-provisioning and rescheduling tasks, so you just sidestep the lifecycle bit of a kubelet upgrade gone pear shaped. I haven't had to touch any of that since February though, but I will be again soon. I hope kubeadm and its ilk have gotten better at handling the lifecycle of a long lived cluster. I've done all that in terraform (from scratch), and I'd really rather have a more standardized option that's not buy $HOSTED or $VENDOR solution.

by merb 8y ago

pm90 8y ago

What did you find complex about updating k8s? Serious question since I work closely with k8s and they have a pretty fast release cycle.

merb 8y ago
running into bugs and that I need to ssh into the nodes ;) (serious answer) I can basically automate nearly everything, os updates (coreos), deployment updates (ci/cd), however updating etcd or k8s is a manual operation on bare metal, either I'm on bootkube than I need to login and update kubectl, than I need to update all daemonsets/deployments in kube-system (it's not always easy and bootkube has strange bugs, https://github.com/kubernetes-incubator/bootkube/issues/977 https://github.com/kubernetes-incubator/bootkube/issues/977) So one should actually use kubeadm, once again I still need to update kubelet, so I need to ssh into the machine (anyway) I also need to update kubeadm and run it, again a manual operation. *ssh means I need to do it on my own, for small clusters this can be done manually for bigger clusters it can be scripted via ansible or even better create an own software that does it. My point is, that's the only thing which is not "just works" on k8s (yet). Edit: I also run into a bug that I could not upgrade k8s with kubeadm (but thats gonna be fixed soon: https://github.com/kubernetes/kubeadm/issues/727 https://github.com/kubernetes/kubeadm/issues/727) Edit2: the good thing is that reloading kubelet won't kill anything.
- barrkel 8y ago
  I'd look at using a regular configuration management system for maintaining k8s itself, e.g. Puppet.
vetrom 8y ago
Similar to a couple of the other replies, updating kubelet or the like tends to touch enough dependencies that it starts looking sensible to do updates as rolling VM re-provisioning and rescheduling tasks, so you just sidestep the lifecycle bit of a kubelet upgrade gone pear shaped. I haven't had to touch any of that since February though, but I will be again soon. I hope kubeadm and its ilk have gotten better at handling the lifecycle of a long lived cluster. I've done all that in terraform (from scratch), and I'd really rather have a more standardized option that's not buy $HOSTED or $VENDOR solution.