5 ms·

I'm still stuck on "Yeah, but I need a credential to authenticate with Vault to get the other credential that I want". Very chicken and egg.

infamousjoeg 8y ago
That was my first thought, too. "But... there's a root account... creating another account... which is typically a privileged action. What's protecting that? Is that root account being rotated, too?"
- Flenser 8y ago
  the master key is split into shards. See under "Why" here: https://www.vaultproject.io/docs/concepts/seal.html https://www.vaultproject.io/docs/concepts/seal.html
  - infamousjoeg 8y ago
    Sure, the master key is split into shards... I'm talking about the account creating the database user. Let's take MSSQL, for example. The equivalent to a root account there is `sa`. So, Vault will have control of the `sa` account in order to create leased database users. If I'm a malicious actor inside the environment, what's stopping me from compromising the `sa` account and mimicking dynamic secrets? I'd need to be comparing Vault logs with Database logs constantly to ensure it was legit. It's just madness, if you ask me.
    - zie 8y ago
      In some cases, vault can change the password of the root account on the service, so that only vault is known to have the password. But you are correct, there is a problem here. There is nothing stopping you from attacking the DB or service directly. You can do a lot to minimize that, many DB auth systems let you specify where connections for users are allowed to come from, only allow a single concurrent connection, etc. Plus you can encrypt and verify that connection with TLS, build up FW rules, etc. Security is never "perfect" but to say dynamic secrets are wrong and crazy just because of this one problem isn't very well thought out. Your wordpress application will be hacked a LOT easier than your MSSQL sa account. So dynamic secret people are saying, we all know our apps leak/get compromised all the time, minimize their leakage with a short lived, one-time use secret, that can only be used from X machine, etc, instead of a long-term, static secret that can be used anywhere, etc. For sure, for high-value targets you best be auditing everything you care about, not just vault, but with a vault audit log, you can automate a lot of that comparison between MSSQL and vault, where before it was much harder to automate since the logs from every application using MSSQL will be different, etc. But if you are in a high-value org like this, you are almost certainly already auditing your MSSQL account creations, this doesn't change that.
    - maccam94 8y ago
      I think the solution to this problem is to not run your apps with the root account in the database. Then the app db passwords can be automatically rotated by vault, and db admins can request temporary root access from vault if necessary (and those credentials should be short-lived).
- mjlee 8y ago
  That's protected by Shamir's Secret Sharing the root encryption key, which adds a fairly robust layer of security. It means that (unless you store the key parts in an HSM/Cloud KMS) there's a trade off of having a manual step to get the unencrypted key in to vault's memory.
  - drablyechoes 8y ago
    You can also set up vault to encrypt each shard of the root key with the GPG pubkey for each administrator before dumping them to output. This way, plaintext shards of the root key never even touch the disk or otherwise come into view of anybody other than the intended recipient.
kgilpin 8y ago
Yes, establishing that “secret 0” (the credential which grants access to all the others) is a tough problem. In some scenarios it’s possible to at least avoid introducing another vault-specific credential. For example if your workload is running in a managed environment like AWS or Kubernetes, the platform provides a credential automatically (IAM role or service account). If the vault can recognize this platform credential and map it to a role defined in the vault, then you are using the platform credential as the “secret 0”. It’s still not completely idiot proof because you have to define the mapping between platform roles and vault roles, and define permissions between vault roles and secrets. And of course the detailed workflows for doing this vary between vaults.
vinceguidry 8y ago
Vault doesn't try to solve the human trust problem, just the application trust problem. It assumes you can trust the humans involved in the system. Secrets simply don't live with applications. Solving the human trust problem is far more difficult and outside the realm of a single software application.
- protonfish 8y ago
  But if the human "trust" problem is the weakest link, improving security in other ways is just theater. It's like adding a deadbolt to a door adjacent to an open window.
  - vinceguidry 8y ago
    The human trust problem isn't the weakest link, the application is. Way more applications get hacked than humans.
tonyarkles 8y ago
One situation where I've deployed Vault, it was configured to issue the Vault credential using LDAP (Active Directory, in that case) in the "user needs a credential for a particular system" case. We didn't do fine-grained permissions for that, but it provided the audit trail we needed. In the automated case, Hashicorp Nomad can issue tokens to individual applications. I was working on that but left the company before getting it into production. That does require trusting your Nomad infrastructure with long-term tokens, but that's still quite a bit better than, e.g., committing tokens into Git :)
armon 8y ago
There are some good suggestions in the replies here, but I'd recommend looking at our secure introduction guide here: https://www.vaultproject.io/guides/identity/secure-intro.html https://www.vaultproject.io/guides/identity/secure-intro.htm... Effectively, there are only two possible approaches. Platform specific integrations, such as AWS IAM, Kubernetes, Nomad, etc. Or Trusted Orchestrators, which help inject the initial "secret zero".
- peteski22 8y ago
  Hi Armon, Under the 'trusted orchestrator' model the article you linked to describes it states "you have an orchestrator which is already authenticated against Vault with privileged permissions". https://www.vaultproject.io/guides/identity/secure-intro.html#trusted-orchestrator https://www.vaultproject.io/guides/identity/secure-intro.htm... In the case of the AppRole auth method and where the orchestrator is an automated app, would it be fair to suggest that you'd have an AppRole for which secret IDs are generated that do not expire based on time and/or max uses? From the Vault documentation here: https://www.vaultproject.io/api/auth/approle/index.html#create-new-approle https://www.vaultproject.io/api/auth/approle/index.html#crea... Is it possible to set 'secret_id_ttl' to something like '0' so that it never expires? The reason behind my question is that if the automated app is initially seeded with a role and secret ID it can login to Vault and get a token, which can be renewed going forward (based on settings for the AppRole). However, if the token is not renewed in time, or the service has to be restarted, you would need to regenerate the secret ID and reseed the application with it, which would be a manual process. Thanks for any advice :)
  - armon 8y ago
    Hey peteski22, Exactly what you suggested would work! Having an AppRole that never expires would allow the trusted orchestrator to authenticate on each run, and then generate and inject ephemeral credentials.

infamousjoeg 8y ago

That was my first thought, too. "But... there's a root account... creating another account... which is typically a privileged action. What's protecting that? Is that root account being rotated, too?"

Flenser 8y ago
the master key is split into shards. See under "Why" here: https://www.vaultproject.io/docs/concepts/seal.html https://www.vaultproject.io/docs/concepts/seal.html
- infamousjoeg 8y ago
  Sure, the master key is split into shards... I'm talking about the account creating the database user. Let's take MSSQL, for example. The equivalent to a root account there is `sa`. So, Vault will have control of the `sa` account in order to create leased database users. If I'm a malicious actor inside the environment, what's stopping me from compromising the `sa` account and mimicking dynamic secrets? I'd need to be comparing Vault logs with Database logs constantly to ensure it was legit. It's just madness, if you ask me.
  - zie 8y ago
    In some cases, vault can change the password of the root account on the service, so that only vault is known to have the password. But you are correct, there is a problem here. There is nothing stopping you from attacking the DB or service directly. You can do a lot to minimize that, many DB auth systems let you specify where connections for users are allowed to come from, only allow a single concurrent connection, etc. Plus you can encrypt and verify that connection with TLS, build up FW rules, etc. Security is never "perfect" but to say dynamic secrets are wrong and crazy just because of this one problem isn't very well thought out. Your wordpress application will be hacked a LOT easier than your MSSQL sa account. So dynamic secret people are saying, we all know our apps leak/get compromised all the time, minimize their leakage with a short lived, one-time use secret, that can only be used from X machine, etc, instead of a long-term, static secret that can be used anywhere, etc. For sure, for high-value targets you best be auditing everything you care about, not just vault, but with a vault audit log, you can automate a lot of that comparison between MSSQL and vault, where before it was much harder to automate since the logs from every application using MSSQL will be different, etc. But if you are in a high-value org like this, you are almost certainly already auditing your MSSQL account creations, this doesn't change that.
  - maccam94 8y ago
    I think the solution to this problem is to not run your apps with the root account in the database. Then the app db passwords can be automatically rotated by vault, and db admins can request temporary root access from vault if necessary (and those credentials should be short-lived).
mjlee 8y ago
That's protected by Shamir's Secret Sharing the root encryption key, which adds a fairly robust layer of security. It means that (unless you store the key parts in an HSM/Cloud KMS) there's a trade off of having a manual step to get the unencrypted key in to vault's memory.
- drablyechoes 8y ago
  You can also set up vault to encrypt each shard of the root key with the GPG pubkey for each administrator before dumping them to output. This way, plaintext shards of the root key never even touch the disk or otherwise come into view of anybody other than the intended recipient.

kgilpin 8y ago

Yes, establishing that “secret 0” (the credential which grants access to all the others) is a tough problem. In some scenarios it’s possible to at least avoid introducing another vault-specific credential. For example if your workload is running in a managed environment like AWS or Kubernetes, the platform provides a credential automatically (IAM role or service account). If the vault can recognize this platform credential and map it to a role defined in the vault, then you are using the platform credential as the “secret 0”. It’s still not completely idiot proof because you have to define the mapping between platform roles and vault roles, and define permissions between vault roles and secrets. And of course the detailed workflows for doing this vary between vaults.

vinceguidry 8y ago

Vault doesn't try to solve the human trust problem, just the application trust problem. It assumes you can trust the humans involved in the system. Secrets simply don't live with applications. Solving the human trust problem is far more difficult and outside the realm of a single software application.

protonfish 8y ago
But if the human "trust" problem is the weakest link, improving security in other ways is just theater. It's like adding a deadbolt to a door adjacent to an open window.
- vinceguidry 8y ago
  The human trust problem isn't the weakest link, the application is. Way more applications get hacked than humans.

tonyarkles 8y ago

One situation where I've deployed Vault, it was configured to issue the Vault credential using LDAP (Active Directory, in that case) in the "user needs a credential for a particular system" case. We didn't do fine-grained permissions for that, but it provided the audit trail we needed. In the automated case, Hashicorp Nomad can issue tokens to individual applications. I was working on that but left the company before getting it into production. That does require trusting your Nomad infrastructure with long-term tokens, but that's still quite a bit better than, e.g., committing tokens into Git :)

armon 8y ago

There are some good suggestions in the replies here, but I'd recommend looking at our secure introduction guide here: https://www.vaultproject.io/guides/identity/secure-intro.html https://www.vaultproject.io/guides/identity/secure-intro.htm... Effectively, there are only two possible approaches. Platform specific integrations, such as AWS IAM, Kubernetes, Nomad, etc. Or Trusted Orchestrators, which help inject the initial "secret zero".

peteski22 8y ago
Hi Armon, Under the 'trusted orchestrator' model the article you linked to describes it states "you have an orchestrator which is already authenticated against Vault with privileged permissions". https://www.vaultproject.io/guides/identity/secure-intro.html#trusted-orchestrator https://www.vaultproject.io/guides/identity/secure-intro.htm... In the case of the AppRole auth method and where the orchestrator is an automated app, would it be fair to suggest that you'd have an AppRole for which secret IDs are generated that do not expire based on time and/or max uses? From the Vault documentation here: https://www.vaultproject.io/api/auth/approle/index.html#create-new-approle https://www.vaultproject.io/api/auth/approle/index.html#crea... Is it possible to set 'secret_id_ttl' to something like '0' so that it never expires? The reason behind my question is that if the automated app is initially seeded with a role and secret ID it can login to Vault and get a token, which can be renewed going forward (based on settings for the AppRole). However, if the token is not renewed in time, or the service has to be restarted, you would need to regenerate the secret ID and reseed the application with it, which would be a manual process. Thanks for any advice :)
- armon 8y ago
  Hey peteski22, Exactly what you suggested would work! Having an AppRole that never expires would allow the trusted orchestrator to authenticate on each run, and then generate and inject ephemeral credentials.