3 ms·
My #1 k8s nightmare is the widespread practice of just writing (or downloading and never even looking at!) YAML and applying it to the cluster, with no addition
by jhugo 4y ago
My #1 k8s nightmare is the widespread practice of just writing (or downloading and never even looking at!) YAML and applying it to the cluster, with no additional management layer (we use Terraform, but use whatever you want), meaning that eventually you have no idea what the intended state of the cluster is, only its actual state. Vendor READMEs encourage this (some even going so far as to suggest `kubectl apply -f https:// https://...`!).
My #2 (probably partially caused by #1) is the lack of attention paid to RBAC in vendor-supplied manifests. Multiple times I've found that the vendor's YAML binds some controller's service account to a ClusterRole giving access to all secrets in the cluster, when it only really needs to read one. After filing a GitHub issue it seems that I'm the first to even notice, even on popular projects that have been around for years.
- nijave 4y ago>Multiple times I've found that the vendor's <software does something stupid> ...sounds about right (at least for closed source products)
- jhugo 4y agoNo experience with deploying anything from a closed source vendor on k8s — these experiences have been exclusively with OSS. Cluster-wide secret access is one of the worst I've come across, but smaller problems are virtually universal. We've come to see the YAML shipped by projects as an example, even when they document it as the preferred installation method. We always write our own now. Even shipped Helm charts are no better, they usually encapsulate the same problems but just make them harder to fix yourself (since you are incentivised not to fork the chart as you'll have to maintain it).