4 ms·

First, thanks for taking the time to offer feedback regarding what we call "secrets" in Kubernetes. As you've rightfully pointed out the current implementation

First, thanks for taking the time to offer feedback regarding what we call "secrets" in Kubernetes. As you've rightfully pointed out the current implementation of "secrets" are stored unencrypted in etcd; something like Hashicorp's Vault [1] would be a much better choice for the given use-case. I've already began exploring how an integration between Vault and Kubernetes would work [2]. There are some things to work out, but those discussions are under way [3]. The current prototype/example demonstrates one way of leveraging Vault from applications deployed on Kubernetes, without changes to the Kubernetes core. While the vault-controller [2] works, we can do better. One idea is to consider what a deeper Vault integration looks like. Ideally we can modify parts of Kubernetes, mainly the kubelet agent that runs on every node, to handled the secure generation and renewal of unique Vault tokens for each Pod during the Pod creation phase. Another idea would be to rethink "secrets" altogether and rebuild the entire feature on top of the Vault API. We can "hide" Vault under the current "secrets" API, which would let us remain backwards compatible. That would be phase one. Phase two could introduce new Secret extensions, which would enable users to declaratively manage Vault tokens, backends, and secret renewals through the Kubernetes API. [1] https://www.vaultproject.io https://www.vaultproject.io [2] https://github.com/kelseyhightower/vault-controller https://github.com/kelseyhightower/vault-controller [3] https://github.com/kubernetes/kubernetes/issues/10439#issuecomment-263954184 https://github.com/kubernetes/kubernetes/issues/10439#issuec...

otterley 10y ago
I appreciate your efforts here, Kelsey. What I'm looking for (and I realize I'm not your manager; this is just my opinion borne of a year of experience with K8S) is a focus on strengthening the core first, then going back to the veneer. Great edifices start with a strong foundation. Also, the engineering teams need to do a better job of integrating community PRs that address implementor demands. For example, Consul K/V support has been asked for since 2014 [1], yet nobody seems to be in a hurry to integrate the (generously-provided) functionality [2]. Finally, I'd like to see deadlines and decision-makers appointed for making decisions such as the one you discuss, so we can avoid endless debates and make forward progress quickly. [1] https://github.com/kubernetes/kubernetes/issues/1957 https://github.com/kubernetes/kubernetes/issues/1957 [2] https://github.com/kubernetes/kubernetes/pull/31622 https://github.com/kubernetes/kubernetes/pull/31622
- kelseyhightower 10y ago
  We all want a strong foundation and will continue to work to create one. In parallel, many of us like building and sharing things, and given that most things in Kubernetes are just abstractions, the things we are building today will continue to work tomorrow, even if the "secrets" backend is reimplemented on top of something like Vault. I'm hopeful people find things like konfd useful, even if you only use configmaps and avoid secrets entirely -- a use case supported by konfd.
  - Terretta 10y ago
    Also consider Consul/Vault so you can leverage the remarkably uncoupled portability of envconsul: https://github.com/hashicorp/envconsul https://github.com/hashicorp/envconsul // PS. To help avoid confusion, I wish K8s (not really) secrets examples didn't use the word "vault" unless using, you know, "vault".

otterley 10y ago

I appreciate your efforts here, Kelsey. What I'm looking for (and I realize I'm not your manager; this is just my opinion borne of a year of experience with K8S) is a focus on strengthening the core first, then going back to the veneer. Great edifices start with a strong foundation. Also, the engineering teams need to do a better job of integrating community PRs that address implementor demands. For example, Consul K/V support has been asked for since 2014 [1], yet nobody seems to be in a hurry to integrate the (generously-provided) functionality [2]. Finally, I'd like to see deadlines and decision-makers appointed for making decisions such as the one you discuss, so we can avoid endless debates and make forward progress quickly. [1] https://github.com/kubernetes/kubernetes/issues/1957 https://github.com/kubernetes/kubernetes/issues/1957 [2] https://github.com/kubernetes/kubernetes/pull/31622 https://github.com/kubernetes/kubernetes/pull/31622

kelseyhightower 10y ago
We all want a strong foundation and will continue to work to create one. In parallel, many of us like building and sharing things, and given that most things in Kubernetes are just abstractions, the things we are building today will continue to work tomorrow, even if the "secrets" backend is reimplemented on top of something like Vault. I'm hopeful people find things like konfd useful, even if you only use configmaps and avoid secrets entirely -- a use case supported by konfd.
- Terretta 10y ago
  Also consider Consul/Vault so you can leverage the remarkably uncoupled portability of envconsul: https://github.com/hashicorp/envconsul https://github.com/hashicorp/envconsul // PS. To help avoid confusion, I wish K8s (not really) secrets examples didn't use the word "vault" unless using, you know, "vault".