4 ms·

My main concern there would be the management and orchestration of the releases for that since its detached from the codebase that is changing. It sounds very p

My main concern there would be the management and orchestration of the releases for that since its detached from the codebase that is changing. It sounds very performant though. Have any examples of this setup?

derefr 5y ago
I don’t, but that’s because I keep my own infrastructure config living inside the same repo the code lives in. We do k8s GitOps with [a simulacra of] Google Kubernetes Engine’s “Application Delivery” (https://cloud.google.com/kubernetes-engine/docs/concepts/add-on/application-delivery https://cloud.google.com/kubernetes-engine/docs/concepts/add...). In this approach, you keep all your k8s manifests relating to the app in the app’s repo, and then the app is “released” by running a command that does the following: 1. tags a particular commit, which locks down the release as being of a particular codebase + a particular target converged k8s state; 2. generates a Docker image, tags it with the commit tag, and pushes it; 3. makes a copy of the manifests; 4. burns the build-image’s SHA into the copy of the manifests; 5. compiles the source manifests (which are using Kustomize) into a single static manifest, which is a complete definition of the new target converged cluster state for the app's k8s namespace; 6. commits that static manifest to a separate tagged “deployment” repo. A separate "deploy" command is then used to reach out to a cluster-side converger component and tells it to pull a particular commit from the "deployment" repo and converge to it. ----- We do have subcomponents that live outside this repo, though; we manage them by: 1. running a “release” within the subcomponent repo (which does create a git tag + push a build-image tagged with that tag; but doesn’t generate/commit any k8s manifests, since the subcomponent repos don’t have any k8s config of their own); and then 2. manually (for now) taking the git tag of the built image, and updating the main app repo’s k8s per-env Kustomization.yaml file with it, i.e. updating a stanza like this with a new value for "newTag": images: - name: gcr.io/our-project/subcomponent newTag: v20210409094011 In theory, our main component could also be tracked as a subcomponent in this manner, such that all our infra config would live in its own basically-empty "app" repo; but that would lose one of the main advantages of GitOps, which is being able to see from the git log exactly what went into a deployed release, both in terms of build-image and infra-config. Our subcomponent services don't evolve nearly as quickly as our main app does, so we don't lose much in the way of release comprehensibility by having them "symbolically linked" to the release like this. As it is, though, our app's LB config for api.example.com lives as a k8s Ingress manifest at /.appctl/config/base/unified-api/ingress.yaml within our app repo. (We're using ingress-nginx, so there's a pretty direct 1:1 mapping between this manifest and an Nginx server{} block.)

derefr 5y ago

I don’t, but that’s because I keep my own infrastructure config living inside the same repo the code lives in. We do k8s GitOps with [a simulacra of] Google Kubernetes Engine’s “Application Delivery” (https://cloud.google.com/kubernetes-engine/docs/concepts/add-on/application-delivery https://cloud.google.com/kubernetes-engine/docs/concepts/add...). In this approach, you keep all your k8s manifests relating to the app in the app’s repo, and then the app is “released” by running a command that does the following: 1. tags a particular commit, which locks down the release as being of a particular codebase + a particular target converged k8s state; 2. generates a Docker image, tags it with the commit tag, and pushes it; 3. makes a copy of the manifests; 4. burns the build-image’s SHA into the copy of the manifests; 5. compiles the source manifests (which are using Kustomize) into a single static manifest, which is a complete definition of the new target converged cluster state for the app's k8s namespace; 6. commits that static manifest to a separate tagged “deployment” repo. A separate "deploy" command is then used to reach out to a cluster-side converger component and tells it to pull a particular commit from the "deployment" repo and converge to it. ----- We do have subcomponents that live outside this repo, though; we manage them by: 1. running a “release” within the subcomponent repo (which does create a git tag + push a build-image tagged with that tag; but doesn’t generate/commit any k8s manifests, since the subcomponent repos don’t have any k8s config of their own); and then 2. manually (for now) taking the git tag of the built image, and updating the main app repo’s k8s per-env Kustomization.yaml file with it, i.e. updating a stanza like this with a new value for "newTag": images: - name: gcr.io/our-project/subcomponent newTag: v20210409094011 In theory, our main component could also be tracked as a subcomponent in this manner, such that all our infra config would live in its own basically-empty "app" repo; but that would lose one of the main advantages of GitOps, which is being able to see from the git log exactly what went into a deployed release, both in terms of build-image and infra-config. Our subcomponent services don't evolve nearly as quickly as our main app does, so we don't lose much in the way of release comprehensibility by having them "symbolically linked" to the release like this. As it is, though, our app's LB config for api.example.com lives as a k8s Ingress manifest at /.appctl/config/base/unified-api/ingress.yaml within our app repo. (We're using ingress-nginx, so there's a pretty direct 1:1 mapping between this manifest and an Nginx server{} block.)