3 ms·
When you update a Kubernetes deployment in the API directly you're essentially modifying a global variable. If you do this as the result of some imperative proc
by tazjin 9y ago
When you update a Kubernetes deployment in the API directly you're essentially modifying a global variable. If you do this as the result of some imperative process (e.g. a CI pipeline), you don't have any record telling you what the value should be - you only know what it currently is.
You're also only modifying a single piece of your whole state at any given time, meaning that if you have the services A and B and their deployment pipelines independently modify state - you have nothing that declares any relationship between which versions of these should be deployed together.
There's a little piece of infrastructure wisdom I've learned over the years, I refer to it sometimes as "tazjin's law":
Any infrastructure component not controlled by a reconciliation process will eventually fail.
Versions of dependent components will get out of sync, configuration is being deployed independently of the application, and so on.
In order to reconcile your current state with your desired state you must know what your desired state is.
Does that explain it?
- akvadrako 9y agoThat doesn't explain it. You always need some global variable saying what the current state is. In your case, maybe it's the branch pointer in your git tree. In the k8s deployment, it's a tag value.
- tazjin 9y agoNo, those don't contain the same information. Our Kontemplate repository contains the state of an entire cluster, including all versions and all configuration. A tag value is a single piece of mutable data that is in no relation to the other relevant data.
- akvadrako 9y agoThe tag points to the current version of your docker image, which contains the current configuration, except secrets which you don't want in a repo. Older deployments point to the older docker images which contain the older versions. Still don't see any advantage to justify the additional complexity in your system.
- tazjin 9y agoWe have more than one service, more than one environment, and auditability requirements (due to being a financial institution). I find it more complex to try and keep track of remote state modifications than to have a single source of truth, but whatever floats your boat ;-)
- pas 9y agoYou can export k8s state as JSON/YAML and put that into a git repo if you want.
- plange 9y agoI had this exact same problem with the way i & my team approached our kubernetes clusters. I come from a network management background and in that space we all know of a tool called RANCID. A simple tool that logged into your switches/routers, made a dump & removed counters/oscillating data and stored it into CVS. This predates infra-as-code.. it took years before someone wrote a clone that used Git instead of CVS Made something stupidly similar for kubernetes: https://github.com/pieterlange/kube-backup https://github.com/pieterlange/kube-backup