3 ms·

> Basically, the geniuses at Google decided to make clusters insecure by default, because being an admin is, like, hard, man. So guess what? People deployed clu

by znpy 3y ago

> Basically, the geniuses at Google decided to make clusters insecure by default, because being an admin is, like, hard, man. So guess what? People deployed clusters that were insecure by default. There is a lot to unpack here, but while I agree with the main point (kubernetes defauls suck) the main problem I’ve seen all across the industry is that people expect to be able to install stuff around and that to be magically secure, functional and performant. It just doesn’t work like that. Systems administration is its own discipline and you can’t expect to follow some random tutorial off the internet and be ready for production. Also, it doesn’t help that the kubernetes documentation is awful. I remember my time working with kubernetes, it’s a patchwork of stuff… it does require time and a fairly high pain threshold to properly master it.

orthoxerox 3y ago
Well, there are two opposing approaches: "up and running" defaults or production defaults. You choose "up-and-running" defaults if you want people to be able to immediately run your software. So this means no encryption, no auth (or "the first user is the admin" kind of auth), storing your state in a bundled SQLite DB, etc. This helps with growth, because random devs can download the container image, run it and check out your features in five minutes. And then run it in production. With production defaults everything is locked down. Go issue some certificates and make sure your OS (or JRE) trusts them, because your nodes won't talk to each other without TLS. No one has any rights to anything by default, can't even log in until you go and edit authN/Z configuration. The container will kill itself if the env var for the database password is still "change_this!". It's a pain to deploy this, but a pleasure to run. Ideally, all products should come with two sets of defaults. Your AIO quickstart image should be easy to run. Your production-grade image or the configs that come with the distribution should be locked down tighter than Fort Knox. K8s is a piece of infrastructure that doesn't really benefit from an "up and running" approach, but it has this kind of defaults for some reason instead of having every default policy resolve to "not allowed".
- hedora 3y ago
  OpenBSD is an existence proof of a third route: Carefully manage complexity, and write a good manual. OpenBSD is about 22M lines of code; kubernetes is 58M, so the code bases are of comparable complexity (OpenBSD uses many more programming languages than kubernetes does). I'd argue that the base OpenBSD install provides far more utility than Kubernetes, but market share numbers disagree. Sources: https://openhub.net/p/kubernetes https://openhub.net/p/kubernetes https://www.reddit.com/r/openbsd/comments/d9a8c9/how_many_lines_of_code_does_openbsd_65_contain/?rdt=50542 https://www.reddit.com/r/openbsd/comments/d9a8c9/how_many_li... Edit: Using open hub for both says OpenBSD is 75M lines: https://openhub.net/p/openbsd https://openhub.net/p/openbsd
  - lakomen 3y ago
    I don't follow. OpenBSD is an OS, k8s is a set of services running on an OS.
    - code_biologist 3y ago
      Both are things deployed to host network visible services.
- mulmen 3y ago
  > or "the first user is the admin" kind of auth My mind immediately went to Eddie Izzard’s “do you have a flag” bit.
deleted 3y ago
[deleted]
ryukafalz 3y ago
> There is a lot to unpack here, but while I agree with the main point (kubernetes defauls suck) the main problem I’ve seen all across the industry is that people expect to be able to install stuff around and that to be magically secure, functional and performant. > It just doesn’t work like that. Systems administration is its own discipline and you can’t expect to follow some random tutorial off the internet and be ready for production. Disagree. I can install an app to my Sandstorm instance and reasonably expect it to be secure, functional, and performant (inasmuch as the app is performant anyway). The tools we use make things difficult, but I don't think there's as much irreducible complexity here as you're implying. We can and should build better tools.
- mplewis 3y ago
  Sandstorm is a very different category of system from k8s.
  - znpy 3y ago
    I had the same feeling, but frankly I didn’t bother replying because somebody putting sandstorm and kubernetes on the same level shows their complete lack of any understanding whatsoever. Kinda proves my point: some random amateur sysadmin thinking their experience with sandstorm is enough to put kubernetes in production. But thanks for replying, anyway.
    - ryukafalz 3y ago
      I worked with K8s professionally for several years but thanks for assuming I have no idea what I'm talking about. Yes, they're obviously different classes of systems, but not in the ways that matter for this discussion. If the question is whether or not you can design a hosting platform with secure defaults and managed/tightly-scoped communication between applications, the answer is plainly yes. Is Sandstorm as scalable and fault tolerant as Kubernetes? Obviously not.
raesene9 3y ago
Out of curiosity, which bits of the k8s documentation (if there are specific bits) did you find lacking? If it's security related stuff, I can open some issues to see if things can get improved :)
- znpy 3y ago
  K8s documentation falls essentially in two categories: either reference page for k8s components (like the kubelet) that lists all the options woth a very very very brief description … or conceptual tutorial-like that explain some concepts, but without much details. Nothing in the middle. There is little documentation on how to put the various pieces together. That’s all on various random webpages outside of the main kubernetes.io website. The setup tutorial only describes some basic kubeadm-based setup. There’s a lot to improve, if you ask me.

by znpy 3y ago

orthoxerox 3y ago

Well, there are two opposing approaches: "up and running" defaults or production defaults. You choose "up-and-running" defaults if you want people to be able to immediately run your software. So this means no encryption, no auth (or "the first user is the admin" kind of auth), storing your state in a bundled SQLite DB, etc. This helps with growth, because random devs can download the container image, run it and check out your features in five minutes. And then run it in production. With production defaults everything is locked down. Go issue some certificates and make sure your OS (or JRE) trusts them, because your nodes won't talk to each other without TLS. No one has any rights to anything by default, can't even log in until you go and edit authN/Z configuration. The container will kill itself if the env var for the database password is still "change_this!". It's a pain to deploy this, but a pleasure to run. Ideally, all products should come with two sets of defaults. Your AIO quickstart image should be easy to run. Your production-grade image or the configs that come with the distribution should be locked down tighter than Fort Knox. K8s is a piece of infrastructure that doesn't really benefit from an "up and running" approach, but it has this kind of defaults for some reason instead of having every default policy resolve to "not allowed".

hedora 3y ago
OpenBSD is an existence proof of a third route: Carefully manage complexity, and write a good manual. OpenBSD is about 22M lines of code; kubernetes is 58M, so the code bases are of comparable complexity (OpenBSD uses many more programming languages than kubernetes does). I'd argue that the base OpenBSD install provides far more utility than Kubernetes, but market share numbers disagree. Sources: https://openhub.net/p/kubernetes https://openhub.net/p/kubernetes https://www.reddit.com/r/openbsd/comments/d9a8c9/how_many_lines_of_code_does_openbsd_65_contain/?rdt=50542 https://www.reddit.com/r/openbsd/comments/d9a8c9/how_many_li... Edit: Using open hub for both says OpenBSD is 75M lines: https://openhub.net/p/openbsd https://openhub.net/p/openbsd
- lakomen 3y ago
  I don't follow. OpenBSD is an OS, k8s is a set of services running on an OS.
  - code_biologist 3y ago
    Both are things deployed to host network visible services.
mulmen 3y ago
> or "the first user is the admin" kind of auth My mind immediately went to Eddie Izzard’s “do you have a flag” bit.

deleted 3y ago

[deleted]

ryukafalz 3y ago

> There is a lot to unpack here, but while I agree with the main point (kubernetes defauls suck) the main problem I’ve seen all across the industry is that people expect to be able to install stuff around and that to be magically secure, functional and performant. > It just doesn’t work like that. Systems administration is its own discipline and you can’t expect to follow some random tutorial off the internet and be ready for production. Disagree. I can install an app to my Sandstorm instance and reasonably expect it to be secure, functional, and performant (inasmuch as the app is performant anyway). The tools we use make things difficult, but I don't think there's as much irreducible complexity here as you're implying. We can and should build better tools.

mplewis 3y ago
Sandstorm is a very different category of system from k8s.
- znpy 3y ago
  I had the same feeling, but frankly I didn’t bother replying because somebody putting sandstorm and kubernetes on the same level shows their complete lack of any understanding whatsoever. Kinda proves my point: some random amateur sysadmin thinking their experience with sandstorm is enough to put kubernetes in production. But thanks for replying, anyway.
  - ryukafalz 3y ago
    I worked with K8s professionally for several years but thanks for assuming I have no idea what I'm talking about. Yes, they're obviously different classes of systems, but not in the ways that matter for this discussion. If the question is whether or not you can design a hosting platform with secure defaults and managed/tightly-scoped communication between applications, the answer is plainly yes. Is Sandstorm as scalable and fault tolerant as Kubernetes? Obviously not.

raesene9 3y ago

Out of curiosity, which bits of the k8s documentation (if there are specific bits) did you find lacking? If it's security related stuff, I can open some issues to see if things can get improved :)

znpy 3y ago
K8s documentation falls essentially in two categories: either reference page for k8s components (like the kubelet) that lists all the options woth a very very very brief description … or conceptual tutorial-like that explain some concepts, but without much details. Nothing in the middle. There is little documentation on how to put the various pieces together. That’s all on various random webpages outside of the main kubernetes.io website. The setup tutorial only describes some basic kubeadm-based setup. There’s a lot to improve, if you ask me.