6 ms·

Google Kubernetes clusters config checker tool

127 points by mikowhy 4y ago

game-of-throws 4y ago
A similar tool is Popeye. It works on any cluster, not just clusters hosted on GKE. https://github.com/derailed/popeye https://github.com/derailed/popeye
- mikowhy 4y ago
  Our focus is on Google Cloud and Google Kubernetes Engine specifics but I love Popeye and K9S from derailed!
- Bayart 4y ago
  I've used Popeye because it comes shipped with k9s, which is the best k8s "dashboard" in my opinion. I really like it but it tends to bug out a bit on my terminal.
  - mikowhy 4y ago
    they put all the best to k9s: ncurses style of UI + vi style of key bindings
rpadovani 4y ago
Given that is specific for GCP, and developed by somebody within Google, why not including it directly in the webconsole?
- nova22033 4y ago
  This is not an officially supported Google product. How does this work? It's developed by google but it's not officially supported?
  - vaughands 4y ago
    It's likely someone's 20% project.
    - paulgb 4y ago
      Google also has (or had? it's been 7 years since I was there) pretty strict rules around working on your own OSS even on your free time, so the easiest way is (was?) for Google to own the IP and apply an open source license.
      - mdaniel 4y ago
        I somehow got the impression that kind of restriction was illegal in California, although I guess it's like a lot of things in the legal domain: those who have the most lawyers wins
        jkaplowitz 4y ago
        California applies limits to such restrictions, but does allow them when the side projects in question relate to the employer's actual or demonstrably anticipated business, among other exceptions. A tool specific to Google Kubernetes Engine, as we're discussing here, plainly relates to Google's business. My memory of Google US employment legalese - note I have not worked for them for over 7 years and am not speaking for them here - is that they acknowledge limits to their IP assignment provisions which are consistent with California law. Any Googler who is confident that those limits protect their ownership by default of their side project does not have to seek Google's approval in order to own it, even according to the contract wording. But Google's business is so broad that it's often legitimately debatable (and sometimes beyond the knowledge of the Googler doing the work) as to whether something would be in scope. So getting their approval, which can come either with explicit assignment of rights back to the Googler or explicit permission to release under Google copyright, is often the prudent approach to minimize undesired risks.
  - kyrra 4y ago
    (googler, opinions are my own). Google actually publishes most of the docs that Googler's have to follow when open sourcing software. See: https://opensource.google/documentation/reference/releasing https://opensource.google/documentation/reference/releasing This specific line you're asking about is talked about here: https://opensource.google/documentation/reference/releasing/publishing#disclaimer https://opensource.google/documentation/reference/releasing/...
- mikowhy 4y ago
  That is good point but also long way ahead. Sometimes such tools are merging with a core product. The tool and policy was created by Google engineers that are not part of GKE product development teams. For now I suggest to use the tool in a scheduled, serverless manner and configure evaluation output to Security Command Center. By that, processes will be fully automated and the results will be visible in a webconsole (as findings in Security Command Center).
pdimitar 4y ago
As many people predicted years ago (I can't claim to know k8s that well, in fact I suck at it), eventually we'll go full circle and k8s config will just become its own specialized programming language. Maybe we should stop moving these things so agonizingly slow and through the path of natural (and did I mention slow as molasses) evolution and just skip to the endgame that most of us know will inevitably come? Apparently not. I'd like to see Google being more brave here. They are on the forefront of k8s in many ways (or so it seems, maybe I am wrong?). Just make a specialized programming language with a good compiler / linter and let's all collectively be better for it.
- cyberpunk 4y ago
  Why do you need a 'specialized programming language' when kube for most people, is just waiting for various objects via api? I mean, sure, ATM it can be a bit painful to generate your 'Deployment' object in json or yaml or whatever, but then it's just a post to the kubeapi and it's done.. You can do that in any language you want.. What am I missing?
  - pdimitar 4y ago
    You're missing the part where people do this periodically and can never truly remember all the gotchas so they make small mistakes with time that at one point make your cluster fall over with cryptic error messages. I've seen it (but as I said above I am by no means a k8s pro). That's why there are these linters. Having a small super-specialized language that catches errors before "compiling" your configuration to an YAML will help hugely.
    - adamgordonbell 4y ago
      Dhall for kubernetes might be what you are looking for. https://github.com/dhall-lang/dhall-kubernetes https://github.com/dhall-lang/dhall-kubernetes
      - pdimitar 4y ago
solatic 4y ago
https://github.com/google/gke-policy-automation/blob/main/gke-policies/policy/control_plane_access.rego https://github.com/google/gke-policy-automation/blob/main/gk... What's the point of requiring the control plane to be locked down to authorized networks (IP address ranges)? Isn't Google responsible for DDoS protection, enforcing authentication controls (i.e. logging in with a Google account in the right Google group), patching the control plane ASAP for any security vulnerabilities? If you have a VPN, if you have heavy-duty network monitoring on your VPN endpoint, sure, limit it to the VPN. For the rest of us? Is every startup running GKE without heavy-duty VPN / network monitoring fundamentally insecure? That doesn't sound right to me. Security is supposed to be a spectrum, and it seems like black-and-white automated config checkers like these are more likely to provoke arguments internally ("but the tool said it's bad!!") than to help reach a nuanced understanding of why tradeoffs are made. No?
- pid-1 4y ago
  security = onion
- sg1234 4y ago
  Would you feel more comfortable if you had an in cluster agent that dial into a Google Managed API and receives command from it ? This way your control plane can be private and you don't have to handle Managed Authorize Networks.
- no_circuit 4y ago
  Likely security in layers. Why expose your control plane to attacks directly from the internet if you don't have to? Cuts down login attempts noise in logs since anything would have to be coming from the VPC. Other than initial setup of a bastion -- that's the tradeoff -- sounds like less to worry about for a small shop or a startup. Same for Cloud SQL or any other managed service.
  - Ironlink 4y ago
    I wish they would reuse the pattern of Cloud SQL where you can get temporary access without manually handling the Authorized Networks setting. The Cloud SQL API lets you exchange your API access token for a short lived TLS client certificate. This is done client side by things like cloudsql-proxy[1] and the cloud-sql-jdbc-socket-factory java library[2]. This way, I can access my Cloud SQL instance from my IDE, even though my list of authorized networks is empty. I feel like the gke-gcloud-auth-plugin cloud do something very similar. [1]: https://github.com/GoogleCloudPlatform/cloudsql-proxy https://github.com/GoogleCloudPlatform/cloudsql-proxy [2]: https://github.com/GoogleCloudPlatform/cloud-sql-jdbc-socket-factory https://github.com/GoogleCloudPlatform/cloud-sql-jdbc-socket...
zxspectrum1982 4y ago
Red Hat has a similar tool called Red Hat Insights Advisor for OpenShift, which they provide as a free service with OpenShift subscriptions, so you don't need to install anything: What is new in Insights Advisor for Red Hat OpenShift https://www.redhat.com/en/blog/what-new-insights-advisor-openshift https://www.redhat.com/en/blog/what-new-insights-advisor-ope... https://console.redhat.com/openshift/insights/advisor/recommendations https://console.redhat.com/openshift/insights/advisor/recomm...

127 points by mikowhy 4y ago

A similar tool is Popeye. It works on any cluster, not just clusters hosted on GKE. https://github.com/derailed/popeye https://github.com/derailed/popeye

mikowhy 4y ago
Our focus is on Google Cloud and Google Kubernetes Engine specifics but I love Popeye and K9S from derailed!
Bayart 4y ago
I've used Popeye because it comes shipped with k9s, which is the best k8s "dashboard" in my opinion. I really like it but it tends to bug out a bit on my terminal.
- mikowhy 4y ago
  they put all the best to k9s: ncurses style of UI + vi style of key bindings

rpadovani 4y ago

Given that is specific for GCP, and developed by somebody within Google, why not including it directly in the webconsole?

nova22033 4y ago
This is not an officially supported Google product. How does this work? It's developed by google but it's not officially supported?
- vaughands 4y ago
  It's likely someone's 20% project.
  - paulgb 4y ago
    Google also has (or had? it's been 7 years since I was there) pretty strict rules around working on your own OSS even on your free time, so the easiest way is (was?) for Google to own the IP and apply an open source license.
    - mdaniel 4y ago
      I somehow got the impression that kind of restriction was illegal in California, although I guess it's like a lot of things in the legal domain: those who have the most lawyers wins
      - jkaplowitz 4y ago
        California applies limits to such restrictions, but does allow them when the side projects in question relate to the employer's actual or demonstrably anticipated business, among other exceptions. A tool specific to Google Kubernetes Engine, as we're discussing here, plainly relates to Google's business. My memory of Google US employment legalese - note I have not worked for them for over 7 years and am not speaking for them here - is that they acknowledge limits to their IP assignment provisions which are consistent with California law. Any Googler who is confident that those limits protect their ownership by default of their side project does not have to seek Google's approval in order to own it, even according to the contract wording. But Google's business is so broad that it's often legitimately debatable (and sometimes beyond the knowledge of the Googler doing the work) as to whether something would be in scope. So getting their approval, which can come either with explicit assignment of rights back to the Googler or explicit permission to release under Google copyright, is often the prudent approach to minimize undesired risks.
- kyrra 4y ago
  (googler, opinions are my own). Google actually publishes most of the docs that Googler's have to follow when open sourcing software. See: https://opensource.google/documentation/reference/releasing https://opensource.google/documentation/reference/releasing This specific line you're asking about is talked about here: https://opensource.google/documentation/reference/releasing/publishing#disclaimer https://opensource.google/documentation/reference/releasing/...
mikowhy 4y ago
That is good point but also long way ahead. Sometimes such tools are merging with a core product. The tool and policy was created by Google engineers that are not part of GKE product development teams. For now I suggest to use the tool in a scheduled, serverless manner and configure evaluation output to Security Command Center. By that, processes will be fully automated and the results will be visible in a webconsole (as findings in Security Command Center).

pdimitar 4y ago

As many people predicted years ago (I can't claim to know k8s that well, in fact I suck at it), eventually we'll go full circle and k8s config will just become its own specialized programming language. Maybe we should stop moving these things so agonizingly slow and through the path of natural (and did I mention slow as molasses) evolution and just skip to the endgame that most of us know will inevitably come? Apparently not. I'd like to see Google being more brave here. They are on the forefront of k8s in many ways (or so it seems, maybe I am wrong?). Just make a specialized programming language with a good compiler / linter and let's all collectively be better for it.

cyberpunk 4y ago
Why do you need a 'specialized programming language' when kube for most people, is just waiting for various objects via api? I mean, sure, ATM it can be a bit painful to generate your 'Deployment' object in json or yaml or whatever, but then it's just a post to the kubeapi and it's done.. You can do that in any language you want.. What am I missing?
- pdimitar 4y ago
  You're missing the part where people do this periodically and can never truly remember all the gotchas so they make small mistakes with time that at one point make your cluster fall over with cryptic error messages. I've seen it (but as I said above I am by no means a k8s pro). That's why there are these linters. Having a small super-specialized language that catches errors before "compiling" your configuration to an YAML will help hugely.
  - adamgordonbell 4y ago
    Dhall for kubernetes might be what you are looking for. https://github.com/dhall-lang/dhall-kubernetes https://github.com/dhall-lang/dhall-kubernetes
    - pdimitar 4y ago

solatic 4y ago

https://github.com/google/gke-policy-automation/blob/main/gke-policies/policy/control_plane_access.rego https://github.com/google/gke-policy-automation/blob/main/gk... What's the point of requiring the control plane to be locked down to authorized networks (IP address ranges)? Isn't Google responsible for DDoS protection, enforcing authentication controls (i.e. logging in with a Google account in the right Google group), patching the control plane ASAP for any security vulnerabilities? If you have a VPN, if you have heavy-duty network monitoring on your VPN endpoint, sure, limit it to the VPN. For the rest of us? Is every startup running GKE without heavy-duty VPN / network monitoring fundamentally insecure? That doesn't sound right to me. Security is supposed to be a spectrum, and it seems like black-and-white automated config checkers like these are more likely to provoke arguments internally ("but the tool said it's bad!!") than to help reach a nuanced understanding of why tradeoffs are made. No?

pid-1 4y ago
security = onion
sg1234 4y ago
Would you feel more comfortable if you had an in cluster agent that dial into a Google Managed API and receives command from it ? This way your control plane can be private and you don't have to handle Managed Authorize Networks.
no_circuit 4y ago
Likely security in layers. Why expose your control plane to attacks directly from the internet if you don't have to? Cuts down login attempts noise in logs since anything would have to be coming from the VPC. Other than initial setup of a bastion -- that's the tradeoff -- sounds like less to worry about for a small shop or a startup. Same for Cloud SQL or any other managed service.
- Ironlink 4y ago
  I wish they would reuse the pattern of Cloud SQL where you can get temporary access without manually handling the Authorized Networks setting. The Cloud SQL API lets you exchange your API access token for a short lived TLS client certificate. This is done client side by things like cloudsql-proxy[1] and the cloud-sql-jdbc-socket-factory java library[2]. This way, I can access my Cloud SQL instance from my IDE, even though my list of authorized networks is empty. I feel like the gke-gcloud-auth-plugin cloud do something very similar. [1]: https://github.com/GoogleCloudPlatform/cloudsql-proxy https://github.com/GoogleCloudPlatform/cloudsql-proxy [2]: https://github.com/GoogleCloudPlatform/cloud-sql-jdbc-socket-factory https://github.com/GoogleCloudPlatform/cloud-sql-jdbc-socket...

zxspectrum1982 4y ago

Red Hat has a similar tool called Red Hat Insights Advisor for OpenShift, which they provide as a free service with OpenShift subscriptions, so you don't need to install anything: What is new in Insights Advisor for Red Hat OpenShift https://www.redhat.com/en/blog/what-new-insights-advisor-openshift https://www.redhat.com/en/blog/what-new-insights-advisor-ope... https://console.redhat.com/openshift/insights/advisor/recommendations https://console.redhat.com/openshift/insights/advisor/recomm...