6 ms·

Kubernetes State Checker

46 points by ceohockey60 6y ago

chrischen 6y ago
For the port checking problem, aren’t named ports a potential best practice/solution?
gpm 6y ago
> You might ask, with Kubernetes, don’t you declare the state you want and Kubernetes makes that happen Am I using Kubernetes wrong, or is this not really the case? You say "make this yaml file happen", but any given yaml file is a small fraction of your deployment. It doesn't make all the other yaml files happen. It doesn't make the yaml files you no longer want happening, stop happening. It doesn't even tell you "these things were created by 'old' yaml files" and should be garbage collected (since it doesn't seem to have a sense of old yaml files). It's lifted the scope of commands from one `cp` or `apt-get install` at a time, to one yaml file at a time. But it still feels like I'm managing the state of servers by running a sequence of commands, instead of just declaring what it should be?
- foobarbecue 6y ago
  Theoretically, in k8s, you should deploy once, and then things work. You may be missing parts of the puzzle such as resources like jobs and deployments or tooling like Kustomize or skaffold.
- jpdb 6y ago
  > It doesn't make all the other yaml files happen. It doesn't make the yaml files you no longer want happening, stop happening. It doesn't even tell you "these things were created by 'old' yaml files" and should be garbage collected (since it doesn't seem to have a sense of old yaml files). This is definitely one piece of Kubernetes that is getting a lot of attention recently. The three tools I've been paying attention to are Argo CD[0], Flux[1], and Config Sync[2]. All of these allow you to point your repository to a cluster and sync resources from the repo to the cluster, including deletes. [0] https://argoproj.github.io/argo-cd/ https://argoproj.github.io/argo-cd/ [1] https://github.com/fluxcd/flux https://github.com/fluxcd/flux [2] https://cloud.google.com/kubernetes-engine/docs/add-on/config-sync/overview https://cloud.google.com/kubernetes-engine/docs/add-on/confi... Disclaimer: I work at GCP, but not on the GKE team. Opinions are my own.
  - mixedCase 6y ago
    Don't forget Terraform. It's been doing this for me in Kubernetes for a few years now.
    - OJFord 6y ago
      Or helm. (Or helm via terraform.)
  - derefr 6y ago
    As someone within GCP, do you know / can you find out what the story is with Application Delivery (https://cloud.google.com/kubernetes-engine/docs/concepts/add-on/application-delivery https://cloud.google.com/kubernetes-engine/docs/concepts/add...) ? It looks like a promising idea, but it’s half-baked† and showing no signs of progressing to becoming fully-baked. Is it still being actively worked on? I’d love to use it — compared to Flux/Argo, App Delivery has semantics closest to what I want — but only if there’s some promise of it reaching GA and receiving some more polishing. † I could list off the reasons it’s half-baked, but you’re not the target of that ire, so I’ll leave off with it :) There was one blog post about App Delivery (https://cloud.google.com/blog/products/containers-kubernetes/announcing-application-manager-for-google-kubernetes-engine https://cloud.google.com/blog/products/containers-kubernetes...) in early 2020, then nothing. There’s no official site, or reference, for it, or the components within it. The Application Delivery client appctl(1), despite being a standalone static Go binary, isn’t available separately — the only place to get it is from the “pkg” component of gcloud(1). Nor is there any way to deploy the cluster-side components other than through the GKE dashboard/gcloud(1) CLI. So there’s no good way to deploy+test Application Delivery in your own (e.g. minikube) cluster, to test ‘doing GitOps with Application Delivery’ in a sandbox. Nor is there any promise of App Delivery becoming tooling that’s portable to other Kubernetes hosts. Nothing about any of this is open-source or even source-available — despite App Delivery advertising itself as a “best practices” approach to GitOps, something intended to show other vendors how this kind of thing should done. (How are they going to learn GitOps from a black box?) ————— Application Delivery seems to be based on some (Google-internal? Unpublished, at least) software called KALM (which, I assume, translates to “Kubernetes Application Lifecycle Management” or some such.) Installing Application Delivery into a cluster installs “kalm-controller”; and appctl(1) seems to be at-least-in-part a fork or wrapper for a “kalmcli” client. (I learned this by digging through the debugging-symbol table of appctl; it’s not documented anywhere.) It seems that, because of this separation-of-concerns, KALM itself isn’t supposed to be GKE-specific, whereas Application Delivery is. Well, if an open Application Delivery isn’t in the pipeline, is an open KALM? Or was it ever (and did it stall—perhaps because the lead on it left the project?) (I notice that KALM also has something to do with how Google Cloud Marketplace “Click to Deploy” k8s applications are packaged, which at least leads me to believe that unlike Application Delivery itself, KALM is probably still a living project.)
- linkdd 6y ago
  In Kubernetes there is the "Desired State", which is the sum of all resources (your yaml files) and the "Actual State". Whenever a change is requested on the "Desired State", Kubernetes will validate it (using RBAC, Admission Webhooks, ...) before persisting it (in etcd in most cases). Then after the persistance is done, it will notify the operators (watching the resources) that are in charge of performing the side effects (run that docker image, allocate that storage volume, ...), moving the "Actual State" closer to your "Desired State". Creating a new Custom Resource Definition and an operator helps you hide the complexity. For example: - the (builtin) Deployment operator creates a ReplicaSet - the (builtin) ReplicaSet operator creates Pods - the (builtin) Pod operator runs Docker images and updates the Pods status Nothing prevents you from making an operator that creates Deployment/Services/Ingress for you. This is even simplified with Helm (to template your resources) and the HelmOperator[1]. [1] - https://github.com/fluxcd/helm-operator https://github.com/fluxcd/helm-operator
mdaniel 6y ago
export GOPATH=$GOPATH:$PWD Wasn't the need to do that removed by the use of go.mod? Roughly the same question about the 8 or so manual (and unpinned) `go get` commands Pedantically, the command should also be export GOPATH=$GOPATH${GOPATH:+:}$PWD otherwise one will end up with a GOPATH of ":/the/dir"
joshribakoff 6y ago
Can write helm tests which seems like the same thing as this

46 points by ceohockey60 6y ago

For the port checking problem, aren’t named ports a potential best practice/solution?

gpm 6y ago

> You might ask, with Kubernetes, don’t you declare the state you want and Kubernetes makes that happen Am I using Kubernetes wrong, or is this not really the case? You say "make this yaml file happen", but any given yaml file is a small fraction of your deployment. It doesn't make all the other yaml files happen. It doesn't make the yaml files you no longer want happening, stop happening. It doesn't even tell you "these things were created by 'old' yaml files" and should be garbage collected (since it doesn't seem to have a sense of old yaml files). It's lifted the scope of commands from one `cp` or `apt-get install` at a time, to one yaml file at a time. But it still feels like I'm managing the state of servers by running a sequence of commands, instead of just declaring what it should be?

foobarbecue 6y ago
Theoretically, in k8s, you should deploy once, and then things work. You may be missing parts of the puzzle such as resources like jobs and deployments or tooling like Kustomize or skaffold.
jpdb 6y ago
> It doesn't make all the other yaml files happen. It doesn't make the yaml files you no longer want happening, stop happening. It doesn't even tell you "these things were created by 'old' yaml files" and should be garbage collected (since it doesn't seem to have a sense of old yaml files). This is definitely one piece of Kubernetes that is getting a lot of attention recently. The three tools I've been paying attention to are Argo CD[0], Flux[1], and Config Sync[2]. All of these allow you to point your repository to a cluster and sync resources from the repo to the cluster, including deletes. [0] https://argoproj.github.io/argo-cd/ https://argoproj.github.io/argo-cd/ [1] https://github.com/fluxcd/flux https://github.com/fluxcd/flux [2] https://cloud.google.com/kubernetes-engine/docs/add-on/config-sync/overview https://cloud.google.com/kubernetes-engine/docs/add-on/confi... Disclaimer: I work at GCP, but not on the GKE team. Opinions are my own.
- mixedCase 6y ago
  Don't forget Terraform. It's been doing this for me in Kubernetes for a few years now.
  - OJFord 6y ago
    Or helm. (Or helm via terraform.)
- derefr 6y ago
  As someone within GCP, do you know / can you find out what the story is with Application Delivery (https://cloud.google.com/kubernetes-engine/docs/concepts/add-on/application-delivery https://cloud.google.com/kubernetes-engine/docs/concepts/add...) ? It looks like a promising idea, but it’s half-baked† and showing no signs of progressing to becoming fully-baked. Is it still being actively worked on? I’d love to use it — compared to Flux/Argo, App Delivery has semantics closest to what I want — but only if there’s some promise of it reaching GA and receiving some more polishing. † I could list off the reasons it’s half-baked, but you’re not the target of that ire, so I’ll leave off with it :) There was one blog post about App Delivery (https://cloud.google.com/blog/products/containers-kubernetes/announcing-application-manager-for-google-kubernetes-engine https://cloud.google.com/blog/products/containers-kubernetes...) in early 2020, then nothing. There’s no official site, or reference, for it, or the components within it. The Application Delivery client appctl(1), despite being a standalone static Go binary, isn’t available separately — the only place to get it is from the “pkg” component of gcloud(1). Nor is there any way to deploy the cluster-side components other than through the GKE dashboard/gcloud(1) CLI. So there’s no good way to deploy+test Application Delivery in your own (e.g. minikube) cluster, to test ‘doing GitOps with Application Delivery’ in a sandbox. Nor is there any promise of App Delivery becoming tooling that’s portable to other Kubernetes hosts. Nothing about any of this is open-source or even source-available — despite App Delivery advertising itself as a “best practices” approach to GitOps, something intended to show other vendors how this kind of thing should done. (How are they going to learn GitOps from a black box?) ————— Application Delivery seems to be based on some (Google-internal? Unpublished, at least) software called KALM (which, I assume, translates to “Kubernetes Application Lifecycle Management” or some such.) Installing Application Delivery into a cluster installs “kalm-controller”; and appctl(1) seems to be at-least-in-part a fork or wrapper for a “kalmcli” client. (I learned this by digging through the debugging-symbol table of appctl; it’s not documented anywhere.) It seems that, because of this separation-of-concerns, KALM itself isn’t supposed to be GKE-specific, whereas Application Delivery is. Well, if an open Application Delivery isn’t in the pipeline, is an open KALM? Or was it ever (and did it stall—perhaps because the lead on it left the project?) (I notice that KALM also has something to do with how Google Cloud Marketplace “Click to Deploy” k8s applications are packaged, which at least leads me to believe that unlike Application Delivery itself, KALM is probably still a living project.)
linkdd 6y ago
In Kubernetes there is the "Desired State", which is the sum of all resources (your yaml files) and the "Actual State". Whenever a change is requested on the "Desired State", Kubernetes will validate it (using RBAC, Admission Webhooks, ...) before persisting it (in etcd in most cases). Then after the persistance is done, it will notify the operators (watching the resources) that are in charge of performing the side effects (run that docker image, allocate that storage volume, ...), moving the "Actual State" closer to your "Desired State". Creating a new Custom Resource Definition and an operator helps you hide the complexity. For example: - the (builtin) Deployment operator creates a ReplicaSet - the (builtin) ReplicaSet operator creates Pods - the (builtin) Pod operator runs Docker images and updates the Pods status Nothing prevents you from making an operator that creates Deployment/Services/Ingress for you. This is even simplified with Helm (to template your resources) and the HelmOperator[1]. [1] - https://github.com/fluxcd/helm-operator https://github.com/fluxcd/helm-operator

mdaniel 6y ago

export GOPATH=$GOPATH:$PWD Wasn't the need to do that removed by the use of go.mod? Roughly the same question about the 8 or so manual (and unpinned) `go get` commands Pedantically, the command should also be export GOPATH=$GOPATH${GOPATH:+:}$PWD otherwise one will end up with a GOPATH of ":/the/dir"

joshribakoff 6y ago

Can write helm tests which seems like the same thing as this