3 ms·
The bit where a secret gets pasted into the Very Secure system is a clear problem. Because before it was pasted, and while it was being pasted, it's not in that
by tialaramex 7y ago
The bit where a secret gets pasted into the Very Secure system is a clear problem. Because before it was pasted, and while it was being pasted, it's not in that secure system it's on some dev's laptop.
Most of your secrets will be/ should be just random bits maybe in some particular format that was convenient for a particular application e.g. a 4-digit PIN or a sixteen character Hexadecimal string, or 10 characters of A-Za-z0-9
So for these cases there's no reason that secret is ever on a developer's laptop. Best case the developer made a truly random secret, maybe they (like me) keep a set of hexadecimal dice on their desk for random choices. Just as likely it's tainted, the developer ran a randomizer until out popped a string they liked - or even they found one on a web site, or used the same one as in their test environment.
Either way, since what you wanted was random bits it makes sense in most cases (not all cases, obviously a secret key you were sent by somebody else, for example an API key for their system will have to be copied somehow) to have a feature that just spits the right format of random bits into the secure system without any human seeing them at all.
Even better, in cases where it's an option, is not to rely on stored secrets at all. I think Monzo's post is not worrying about this difference, but it can be critical in terms of decisions about debugging to prefer to have entirely ephemeral secrets. When a pod goes away, the ephemeral secrets that pod had vanish with it, and so you aren't storing them anywhere anyway. If they aren't stored, they can't get stolen by anybody and you've got one less thing to go wrong.
- p10jkle 7y ago> The bit where a secret gets pasted into the Very Secure system is a clear problem. Because before it was pasted, and while it was being pasted, it's not in that secure system it's on some dev's laptop. Yep, so we have a different system for securely generated random keys. If its a Twilio API key, it realistically has to pass through a dev laptop, and its not that much of a big deal. If its eg an RSA key, we will generate it on an airgapped laptop, encrypt to a public key, and then we have a Vault plugin that decrypts, and writes it into Vault. So the unencrypted data is never anywhere but the airgapped laptop or Vault. We also try to generate keys inside of Vault where possible, and we generate a lot of certificates this way.
- mitchellh 7y ago(Disclaimer: founder of HashiCorp, creator of Vault) You hit on a good point. Vault has features to eliminate this security risk, if I’m understanding correctly. The first feature is dynamic secrets: this generates an ephemeral, leased set of credentials that are unique per client. For a Kubernetes pod, it would get a unique set of DB credentials, for example. These are tied to the service account (used for auth). When the auth expires, so do the credentials (they’re dropped from the DB, and if the DB supports it we also drop connections). The second feature is root credential rotation. To use the above feature, a user had to at some point “paste” the superuser credentials into Vault. As you pointed out, there’s a risk here. So what Vault can do is _immediately_ rotate that credential so after configuring Vault, it is no longer valid and only Vault knows the real credential. We support this for most database backend, for example. If you combine these two elements, you get fully ephemeral secrets that are unknown by anyone except the necessary user. There’s a lot more we can talk about, there’s a lot more features we have around this, but this is just the high level point!
- p10jkle 7y agoWe are using your database credential generation features! With thousands of clients, too. Very cool, we'll write a blog post soon. The root credential rotation is genius.
- ekidd 7y ago> The first feature is dynamic secrets: this generates an ephemeral, leased set of credentials that are unique per client. Yes, thank you. This is one of those features that once you've lived with it, you can't imagine going back. Essentially, every secret gets an automatic expiration date in the near future. This has several effects: 1. It teaches people to never hard-code secrets anywhere, because they always expire. So people will follow fairly strict credential-management rules out of sheer laziness. 2. It guarantees that you don't have stale secrets lying around in random corners of your company. So even if somebody does record a secret somewhere they shouldn't, the window of attack may only be a few hours.
- SEJeff 7y agoAs one of your commercial customers, I would give you the feedback that there are a lot of features that get closed (on the github vault issue tracker) with terse and not very user friendly reasons. This is off putting to a lot of people. We've recently been escalating via sales for github issues we see use for and hopefully that gets back to product management. Not everyone is able to do that however.
- cs02rm0 7y agoI think there's also an implication here that secrets aren't (generated and) stored on an HSM. I know nothing about banking, but that surprises me.