3 ms·

I think something doesn't quite add up in OP's claim. Having full access to the control plane effectively let's you do anything you want with the cluster. If th

by xg15 3y ago

I think something doesn't quite add up in OP's claim. Having full access to the control plane effectively let's you do anything you want with the cluster. If there were really millions of completely unsecured Kubernetes instances lying around, we should see all kinds of worms and botnets already taking advantage of them - and we should also see lots of attempted probes/attacks to infect new instances as soon as they come online (dark forest, etc). Yet I haven't heard about anything like this so far. This leads me to believe that the "public" instances actually are protected in some way, just not by a private network. Not a Kubernetes expert, but doesn't the Kubeconfig contain some sort of secret that the client needs for authentication? You might be able to get some basic health check without that, but I'd be very surprised if you could do much more without authentication. At least if that's how it works, this would be an example of "zero-trust" networking, where you just assume anything can be reached from the internet anyway and secure your services based on that premise. If done well, it shouldn't matter whether or not your services are actually exposed on the internet or not. Of course in the interest of defense-in-depth, I'd still try to make sure my services are not actually exposed - bugs happen all the time and additional security layer can't do harm.

maxcoder4 3y ago
Of course that's how it works, there are various authentication methods that must be completed [2] to do anything. The author's claim is that similarly like you shouldn't expose RDP to the public internet, you shouldn't expose your k8s API. In practice the difference is that while RDP (or database or WordPress) credentials are vulnerable to bruteforce, in almost all cases k8s clusters are secured by either mtls or jwt tokens - both utterly non bruteforcable. You're not completely right though, unsecured k8s clusters do happen[1]. And there is some defense in depth component, as you've mentioned. And yeah, we're one RCE away from the cloud melting disaster. But then, the same is true for nginx or openssh. [1] http://redhuntlabs.com/blog/unsecured-kubernetes-clusters-exposed/ http://redhuntlabs.com/blog/unsecured-kubernetes-clusters-ex... [2] Fun but meaningless fact: kube-api defaults for both authentication and authorization are to allow anonymous users and allow everything to everyone (AlwaysAllow)[3]. This is meaningless because every k8s distribution in existence changes authorization default with appropriate flags. [3] https://kubernetes.io/docs/reference/access-authn-authz/kubelet-authn-authz/ https://kubernetes.io/docs/reference/access-authn-authz/kube...
- linuxdude314 3y ago
  This is the way. Even in a zero trust environment, it’s wise to have your cluster endpoints internal to a VPC. Zero trust doesn’t mean no internal network, but rather no internal trusted zone. It’s still a great idea to minimize the “blast radius” where you can.
- Repulsion9513 3y ago
  In practice, no one's actually bruteforcing your RDP or database or WordPress. They're using leaked (or common) credentials - which is still a threat to any other type of service. (That, or they exploit a history of vulnerabilities in the software behind things like RDP or databases - and you should assume that all of the software you're using has vulnerabilities... which are most severe in highly trusted systems like a control plane...) OpenSSH is carefully designed with security in mind, far more widely used than Kubernetes, with a fairly minimized attack surface. Nginx is probably a bit less carefully designed, but also doesn't generally have full access to the entire system. Nginx needs to be open to the Internet at large to work (assuming it's running a public website or something). And you probably need some way to manage it from the Internet. I'd say SSH is a pretty good choice there, especially over Kubernetes.
selfmodruntime 3y ago
There already is malware focusing on unsecured docker daemon's hoisted onto a tcp port [1] [1]: https://sysdig.com/blog/zoom-into-kinsing-kdevtmpfsi/ https://sysdig.com/blog/zoom-into-kinsing-kdevtmpfsi/
tzakrajs 3y ago
Back in 2020 I once accidentally kept my personal k8s api open and got hit by a drive-by k8s-in-k8s bitcoin miner attack, it deployed an operator and it failed to run correctly but it created a ton of pods in my default namespace.

by xg15 3y ago

maxcoder4 3y ago

Of course that's how it works, there are various authentication methods that must be completed [2] to do anything. The author's claim is that similarly like you shouldn't expose RDP to the public internet, you shouldn't expose your k8s API. In practice the difference is that while RDP (or database or WordPress) credentials are vulnerable to bruteforce, in almost all cases k8s clusters are secured by either mtls or jwt tokens - both utterly non bruteforcable. You're not completely right though, unsecured k8s clusters do happen[1]. And there is some defense in depth component, as you've mentioned. And yeah, we're one RCE away from the cloud melting disaster. But then, the same is true for nginx or openssh. [1] http://redhuntlabs.com/blog/unsecured-kubernetes-clusters-exposed/ http://redhuntlabs.com/blog/unsecured-kubernetes-clusters-ex... [2] Fun but meaningless fact: kube-api defaults for both authentication and authorization are to allow anonymous users and allow everything to everyone (AlwaysAllow)[3]. This is meaningless because every k8s distribution in existence changes authorization default with appropriate flags. [3] https://kubernetes.io/docs/reference/access-authn-authz/kubelet-authn-authz/ https://kubernetes.io/docs/reference/access-authn-authz/kube...

linuxdude314 3y ago
This is the way. Even in a zero trust environment, it’s wise to have your cluster endpoints internal to a VPC. Zero trust doesn’t mean no internal network, but rather no internal trusted zone. It’s still a great idea to minimize the “blast radius” where you can.
Repulsion9513 3y ago
In practice, no one's actually bruteforcing your RDP or database or WordPress. They're using leaked (or common) credentials - which is still a threat to any other type of service. (That, or they exploit a history of vulnerabilities in the software behind things like RDP or databases - and you should assume that all of the software you're using has vulnerabilities... which are most severe in highly trusted systems like a control plane...) OpenSSH is carefully designed with security in mind, far more widely used than Kubernetes, with a fairly minimized attack surface. Nginx is probably a bit less carefully designed, but also doesn't generally have full access to the entire system. Nginx needs to be open to the Internet at large to work (assuming it's running a public website or something). And you probably need some way to manage it from the Internet. I'd say SSH is a pretty good choice there, especially over Kubernetes.

selfmodruntime 3y ago

There already is malware focusing on unsecured docker daemon's hoisted onto a tcp port [1] [1]: https://sysdig.com/blog/zoom-into-kinsing-kdevtmpfsi/ https://sysdig.com/blog/zoom-into-kinsing-kdevtmpfsi/

tzakrajs 3y ago

Back in 2020 I once accidentally kept my personal k8s api open and got hit by a drive-by k8s-in-k8s bitcoin miner attack, it deployed an operator and it failed to run correctly but it created a ton of pods in my default namespace.