4 ms·

If we have to count the exploits in every new thing against some grand total of allowable exploits then there will never be new things. The question was not whe

If we have to count the exploits in every new thing against some grand total of allowable exploits then there will never be new things. The question was not whether k8s added to the universe of exploits, but whether the exploits make it unready for production. Personally I was more bothered by some of the code quality issues than the list of specific high severity exploits. It's a large project and issues like this will be found.

geggam 7y ago
When the complexity of the attack surface gets to the degree of k8s I would say that is a problem. The fact very few and I do mean very few people understand the low level functions going on ( like the multiple layers of nat via iptables ) and they are simply struggling to keep it running its pretty obvious they arent qualified to run this in production. I have been at google HQ in kubernetes discussions and its frightening how little people know about the internals of it. These arent amateurs off the street either.
- geofft 7y ago
  You keep saying that the attack surface is high, but is it higher than all other software we consider suitable for this purpose? Does anyone understand the JVM and servlet containers? Does anyone understand OpenSSL's state machine? Does anyone understand hardware load balancers? Does anyone understand speculative execution? Does anyone understand the Postgres query planner? Does anyone understand all the same-origin policies? Does anyone understand their laptop's power supply? I've seem a lot of people build a lot of successful systems on things they don't know every detail of, even when not knowing those details is quite dangerous. That Kubernetes is yet another one of these building blocks isn't an indictment of Kubernetes, it's an indictment of the compulsion to understand everything.
  - geggam 7y ago
    Problem is you dont replace all of the other vectors with k8s. You add more to them. When the entirety is so complex seasoned engineers shrug when you ask what is wrong with the stack you have a problem
- markbnj 7y ago
  We already depend upon layer after layer of highly complex software. I'd argue that the complexity of k8s is not out of line with its scope. I don't want to get into a debate about specific things like netfilter. Yeah it's an odd setup and full of warts, but it's completely pluggable. On GKE for example you can now run in a mode where the pod networking is handled as a VPC subnet with load balancing directly to pods. And that's sort of the point: it's the maturing abstractions that are valuable, not the specific implementation of a part like networking. As for struggling to run it, our experience has been different. Granted we're a small user. Our largest cluster has just over 100 nodes. Our highest volume service hits about 15k req/sec at peak. We're on GKE which is a well-managed implementation and that also makes it less risky. In two years of production the platform has been extremely reliable. Moreover we've been able to do things that would have been a lot harder before, such as autoscaling the service I mentioned above so that we're not paying for capacity we don't need off peak.
  - geggam 7y ago
    I agree if you are using k8s GKE is the place to be. Let the experts who designed it run it for you, almost like it was planned that way :)

geggam 7y ago

When the complexity of the attack surface gets to the degree of k8s I would say that is a problem. The fact very few and I do mean very few people understand the low level functions going on ( like the multiple layers of nat via iptables ) and they are simply struggling to keep it running its pretty obvious they arent qualified to run this in production. I have been at google HQ in kubernetes discussions and its frightening how little people know about the internals of it. These arent amateurs off the street either.

geofft 7y ago
You keep saying that the attack surface is high, but is it higher than all other software we consider suitable for this purpose? Does anyone understand the JVM and servlet containers? Does anyone understand OpenSSL's state machine? Does anyone understand hardware load balancers? Does anyone understand speculative execution? Does anyone understand the Postgres query planner? Does anyone understand all the same-origin policies? Does anyone understand their laptop's power supply? I've seem a lot of people build a lot of successful systems on things they don't know every detail of, even when not knowing those details is quite dangerous. That Kubernetes is yet another one of these building blocks isn't an indictment of Kubernetes, it's an indictment of the compulsion to understand everything.
- geggam 7y ago
  Problem is you dont replace all of the other vectors with k8s. You add more to them. When the entirety is so complex seasoned engineers shrug when you ask what is wrong with the stack you have a problem
markbnj 7y ago
We already depend upon layer after layer of highly complex software. I'd argue that the complexity of k8s is not out of line with its scope. I don't want to get into a debate about specific things like netfilter. Yeah it's an odd setup and full of warts, but it's completely pluggable. On GKE for example you can now run in a mode where the pod networking is handled as a VPC subnet with load balancing directly to pods. And that's sort of the point: it's the maturing abstractions that are valuable, not the specific implementation of a part like networking. As for struggling to run it, our experience has been different. Granted we're a small user. Our largest cluster has just over 100 nodes. Our highest volume service hits about 15k req/sec at peak. We're on GKE which is a well-managed implementation and that also makes it less risky. In two years of production the platform has been extremely reliable. Moreover we've been able to do things that would have been a lot harder before, such as autoscaling the service I mentioned above so that we're not paying for capacity we don't need off peak.
- geggam 7y ago
  I agree if you are using k8s GKE is the place to be. Let the experts who designed it run it for you, almost like it was planned that way :)