3 ms·

I am actually a bit surprised there are any clusters out there exposing their service ports. Do people not run firewalls in their nodes? Sure, it may cause some

I am actually a bit surprised there are any clusters out there exposing their service ports. Do people not run firewalls in their nodes? Sure, it may cause some headaches due to a range of random IPs and random ports that Kubernetes expects to be able to bind, but I assume any sysadmin will know how to deal with this. I know k3s has the awesome advice of disabling firewalld[1], but presumably the clusters mentioned in this article are largely using kubeadm or some other tool rather than k3s. With that said, I agree that secrets management on k8s is pretty bad (unless you use an external service like Hashicorp Vault, a frequent pattern that I run into with k8s). Even with RBAC and secrets in a namespace being inaccessible from other namespaces, the moment someone has permissions to read secrets from a namespace, its essentially game over for you, as noted in this paragraph. > Another potential attack vector within a Kubernetes cluster is accessing internal services. ... In cases where authentication is required, the credentials can typically be found within the Kubernetes secrets space. [1] https://docs.k3s.io/advanced#red-hat-enterprise-linux--centos https://docs.k3s.io/advanced#red-hat-enterprise-linux--cento...

merb 3y ago
The firewalld is for the trusted zone… and for the apiserver 6443 https port k3s comes with rbac enabled by default so if you want to access it via kubelet you need to open it?! trusted zone is used by overlay networks or node internal stuff (or if you manually put your interface into it)
crabbone 3y ago
Why would you be surprised? In most cases the only thing users need from a Kubernetes cluster is to access it from the outside... like... why wouldn't they expose it to the outside world? -- this is the most useful thing you can do with it. And the range of things you might want to expose... oh, god only knows. Think about such trivial stuff as FTP or iSCSI -- kinda useful, right? -- but Kubernetes rules for port exposure are idiotic, because you cannot expose ranges. And quite a few of popular Internet protocols assume they can allocate whatever port they want and start talking on it. And then you start fighting Kubernetes... and you'll win, but it will be some custom security solution... with the obvious problem of it being a custom security solution. Now, add to this that Kubernetes will typically mess with your iptables and maybe also arptables, and if your firewall is based on eg. iptables, then you might be out of luck using it with Kubernetes, and again, will end up rolling something "special". Obviously, you might also want, say, DNS to work inside your cluster, but that's a separate can of worms.

merb 3y ago

The firewalld is for the trusted zone… and for the apiserver 6443 https port k3s comes with rbac enabled by default so if you want to access it via kubelet you need to open it?! trusted zone is used by overlay networks or node internal stuff (or if you manually put your interface into it)

crabbone 3y ago

Why would you be surprised? In most cases the only thing users need from a Kubernetes cluster is to access it from the outside... like... why wouldn't they expose it to the outside world? -- this is the most useful thing you can do with it. And the range of things you might want to expose... oh, god only knows. Think about such trivial stuff as FTP or iSCSI -- kinda useful, right? -- but Kubernetes rules for port exposure are idiotic, because you cannot expose ranges. And quite a few of popular Internet protocols assume they can allocate whatever port they want and start talking on it. And then you start fighting Kubernetes... and you'll win, but it will be some custom security solution... with the obvious problem of it being a custom security solution. Now, add to this that Kubernetes will typically mess with your iptables and maybe also arptables, and if your firewall is based on eg. iptables, then you might be out of luck using it with Kubernetes, and again, will end up rolling something "special". Obviously, you might also want, say, DNS to work inside your cluster, but that's a separate can of worms.