7 ms·

Looking at some of these encryption-as-a-service providers, I'm a bit confused on one of the selling points. From my limited understanding: with a traditional s

Looking at some of these encryption-as-a-service providers, I'm a bit confused on one of the selling points. From my limited understanding: with a traditional system, you encrypt in the database but your encryption key likely exists on your main server, possibly in an environment variable. Attacker compromises main system and has access to both encrypted data and encryption keys. So, you instead use something like Vault to request an encryption key in real-time from a remote service and thus don't need to store it on your server. So, one of their selling points on their site is that Vault is better because two systems would have to be compromised by an attacker in order to decrypt sensitive data. The part I don't understand though is, if an attacker has compromised my server, could they not just initiate a request to Vault for a decryption key at that point? I feel like I'm missing something because this sounds like it remains a single point of failure.

avar 8y ago
You have N number of encryption keys for different parts of your data, then a single client server can only decrypt a small part of the data. Since you need to request the key each time to decrypt things like key rotation become easier since you can do them centrally. Even if you can decrypt data X on a client node the key you got an hour ago for X doesn't work anymore, you must request a new one. Then the server can do things like impose rate-limits. Maybe you can decrypt A-F, but only at a certain imposed rate, and alarms will be raised if the rate is exceeded.
zimbatm 8y ago
Vault provides an audit log. Since each process gets their own token it allows to correlate which machine/container got compromised. I think we will see more tooling being developed around that in the future to make the search more efficient. The other aspect is that if secret rotation is well exercised, it means that in case of a breach, taking an instance out of commission and rotating all the secrets is not a risky task anymore. If developers copy-and-paste tokens on slack, the window of attack is limited to the TTL of the secret. It also prevents developers from using the same secret for another unrelated service.
jedberg 8y ago
It's part of security in depth. Yes, if they own the server, they could keep requesting decryption keys. The main thing this protects against is someone getting a copy of the encrypted data, then breaking into a server and getting a key that is good forever. By using this system, it renders their copy of the encrypted data useless. Then you have to add in defenses for the active attack, such as rate limits, anomaly detection on access patterns for decryption keys, and the usual host and network based intrusion detection. It's one part of a complete security strategy.
- hrez 8y ago
  > The main thing this protects against is someone getting a copy of the encrypted data, then breaking into a server and getting a key that is good forever. By using this system, it renders their copy of the encrypted data useless. Does it? Is there any example on when encrypted data and a key in ones possession can be prevented from decryption?
  - jcrites 8y ago
    (I haven't used Vault but I’ve used systems that solve related problems, like KMS.) One approach is that the app server doesn’t possess the encryption key at all (or at least not the master key). Instead, it calls a remote service to decrypt each item as needed (or the item-level data key aka envelope encryption key). In this way, an attacker can compromise the entire data set and the app server, but they still can't decrypt the data. They have to maintain ongoing access to a compromised app server in order to decrypt data items one by one, which (i) could take a long time for large data sets (ii) can be slowed down by rate-limiting (iii) runs the risk of being noticed (iv) if detected, attacker's access can be shut off immediately. Additionally, you might manage and monitor your key management servers differently than your app servers; for example, you have the expectation that no one will ever log into key servers routinely, so any interactive access or unexpected running processes can generate alarms. The set of people who have access to key servers is different and much more limited than the set of people who have access to the app server. The key server can run in a different virtual network from the app server while providing extremely limited access to the app server (just e.g. TCP on the one port needed to provide this service). This approach contrasts to schemes where the app server or data store has an encryption key. If the attacker compromises that, they can lift the entire data set and encryption key out of your systems and process it later -- and it's irretrievably gone. With the key server approach, stolen encrypted data provides no value on its own, and the attacker needs ongoing access to the key server to make sense of the data.
dec0dedab0de 8y ago
You're basically right. However, it does mean that your secrets aren't stored on disk, so if your backups or SAN are compromised then you're still good. It is also good for syncing secrets across multiple servers, and for keeping the secret up to date, if you have a password that needs to update nightly or monthly or whatever.
lmilcin 8y ago
When you deploy a traditionally configured application to the server you will have a bazillion credentials to access different stuff like databases, external systems. Think how you are going to secure that key store you need for SSL? What is the point of having it encrypted when the password lies in a configuration file two folders away? Vaults like this allow you to not have that cryptographic material on the server. Now, this doesn't solve all problems. As you mentioned, when you have breached server security you have access to everything. But, there are other problematic situations that this helps solve. Now you don't have to provide secrets when you are building images and you will not have secrets on the filesystem when somebody only gains partial access that lets him only read files. This also lets you see who and when is accessing secrets and lets you manage secrets (think how you are going to replace database password?)
foobiekr 8y ago
You put the service account token on the filesystem and then have it removed from the filesystem after loading it. then it lives inside a process, where it is still recoverable by a determined adversary who attempts to obtain root and can inspect process memory, but you can further protect that by encrypting it in memory with an ephemeral key (which also should not be at a predictable location).
csours 8y ago
Very simply, it removes the possibility of keys showing up in config in source control or in server backups.
- nineteen999 8y ago
  I like Ansible's approach to this better. We keep all our secrets in ansible-vault and have a password to unlock it for those "who need to know". Sure, this means provisioning VM's requires human interaction to unlock the vault (once) during a kickstart. So what. I'd rather do that than have plaintext or even pre-hashed passwords lying around in source control. As others have pointed out, Hashicorp Vault seems like a good solution to the problems of another Hashicorp product ... Terraform, which used to keep passwords in its state files (not sure if it still does).
  - athenot 8y ago
    You can also configure Ansible to lookup to Hashicorp Vault on the fly. https://docs.ansible.com/ansible/2.5/plugins/lookup/hashi_vault.html https://docs.ansible.com/ansible/2.5/plugins/lookup/hashi_va...
    - nineteen999 8y ago
      Unfortunately it wouldn't work for our network even if I wanted to use it, since our systems aren't allowed to connect outbound to the Internet (for good reason). I suppose I could set up a Vault server locally (?), but it's complete overkill for a problem we already solved in a simpler way. Ansible is gradually adding modules for about everything. It's come along way since I started using it back in the 1.8 or 1.9 days.
      - jyounker 8y ago
        Vault is usually hosted in-house. Vault offers features such as short-term secrets which are unique to each client. Vault itself manages creation and destruction of the credentials on the server, allowing it to enforce credential lifetimes
tetha 8y ago
You're looking at a bad example. Good backup encryption generates a local AES key on a ramdisk, encrypts the backup with that, decrypts the AES key with a public key and stores the result. This makes it cryptographically hard to access the backups. The host never has access to the private key to decrypt the AES key to encrypt the backup into a usable form. There's no vault involved there because public keys aren't security relevant. On the other hand, we're running a lot of mutual TLS authentication via CAs in vault. Vault has the private key of these CAs, and authenticated hosts can request signed client certificates for these CA chains. In this case, vault enforces certificate parameters, TTLs, CLR and other things. You could keep requesting certs, but once I revoke the vault authentication you stole, you're done once the certificate expires within 3 days. And on top of that - yes, it's possible to escalate to vault using local credentials for vault. However, that's a very different threat level. Botnets won't abuse your vault instance. Heck, script kiddies wouldn't know how to abuse your vault instance after getting shell access. If someone can RCE into a box, and find your vault integration, and abuse your vault integration, you're in for a ride and usually, vault won't be your problem at this point.
nightbrawler 8y ago
We use Vault in our environment and deployments... Vault has a concept of using tokens for authentication to pull secrets. You can assign max number of uses, max TTL and other parameters when you generate them. For our deployments, we generate a single token with a max # of uses that matches the target server count for our deploy and also a very short TTL of 5 seconds. Our code gets pushed and the token is passed to each server (in an env var) during that time and a command is passed to refresh environment vars with secrets from Vault. So if a token gets compromised it's very likely to be used up and/or expired.
shawnz 8y ago
In addition to what others have said, having a secrets management system like this makes re-keying easier in the event of a compromise. If one app is compromised, you can easily update the DB keys without having to recompile/redeploy every app that uses the DB. You would still have to redeploy the compromised app to change its keys for accessing the secrets management service, though.
atmosx 8y ago
Vault provides telemetry. The secret request patterns for each service are available. Depending on complexity,anomaly detection can be build on top.
zxcvbn4038 8y ago
The real utility of these products is in generating dynamic short lived credentials. The longer a set of credentials exist the greater chance they end up written down or widely distributednor going home with a former employee. With something like Vault your credentials have a lifespan of days, hours, minutes, whatever the shortest timespan your comfortable with is. That limits your attack surface greatly. If your doing a break glass procedure you can be assured that credentials are revoked after a reasonable amount of time.

avar 8y ago

You have N number of encryption keys for different parts of your data, then a single client server can only decrypt a small part of the data. Since you need to request the key each time to decrypt things like key rotation become easier since you can do them centrally. Even if you can decrypt data X on a client node the key you got an hour ago for X doesn't work anymore, you must request a new one. Then the server can do things like impose rate-limits. Maybe you can decrypt A-F, but only at a certain imposed rate, and alarms will be raised if the rate is exceeded.

zimbatm 8y ago

Vault provides an audit log. Since each process gets their own token it allows to correlate which machine/container got compromised. I think we will see more tooling being developed around that in the future to make the search more efficient. The other aspect is that if secret rotation is well exercised, it means that in case of a breach, taking an instance out of commission and rotating all the secrets is not a risky task anymore. If developers copy-and-paste tokens on slack, the window of attack is limited to the TTL of the secret. It also prevents developers from using the same secret for another unrelated service.

jedberg 8y ago

It's part of security in depth. Yes, if they own the server, they could keep requesting decryption keys. The main thing this protects against is someone getting a copy of the encrypted data, then breaking into a server and getting a key that is good forever. By using this system, it renders their copy of the encrypted data useless. Then you have to add in defenses for the active attack, such as rate limits, anomaly detection on access patterns for decryption keys, and the usual host and network based intrusion detection. It's one part of a complete security strategy.

hrez 8y ago
> The main thing this protects against is someone getting a copy of the encrypted data, then breaking into a server and getting a key that is good forever. By using this system, it renders their copy of the encrypted data useless. Does it? Is there any example on when encrypted data and a key in ones possession can be prevented from decryption?
- jcrites 8y ago
  (I haven't used Vault but I’ve used systems that solve related problems, like KMS.) One approach is that the app server doesn’t possess the encryption key at all (or at least not the master key). Instead, it calls a remote service to decrypt each item as needed (or the item-level data key aka envelope encryption key). In this way, an attacker can compromise the entire data set and the app server, but they still can't decrypt the data. They have to maintain ongoing access to a compromised app server in order to decrypt data items one by one, which (i) could take a long time for large data sets (ii) can be slowed down by rate-limiting (iii) runs the risk of being noticed (iv) if detected, attacker's access can be shut off immediately. Additionally, you might manage and monitor your key management servers differently than your app servers; for example, you have the expectation that no one will ever log into key servers routinely, so any interactive access or unexpected running processes can generate alarms. The set of people who have access to key servers is different and much more limited than the set of people who have access to the app server. The key server can run in a different virtual network from the app server while providing extremely limited access to the app server (just e.g. TCP on the one port needed to provide this service). This approach contrasts to schemes where the app server or data store has an encryption key. If the attacker compromises that, they can lift the entire data set and encryption key out of your systems and process it later -- and it's irretrievably gone. With the key server approach, stolen encrypted data provides no value on its own, and the attacker needs ongoing access to the key server to make sense of the data.

dec0dedab0de 8y ago

You're basically right. However, it does mean that your secrets aren't stored on disk, so if your backups or SAN are compromised then you're still good. It is also good for syncing secrets across multiple servers, and for keeping the secret up to date, if you have a password that needs to update nightly or monthly or whatever.

lmilcin 8y ago

When you deploy a traditionally configured application to the server you will have a bazillion credentials to access different stuff like databases, external systems. Think how you are going to secure that key store you need for SSL? What is the point of having it encrypted when the password lies in a configuration file two folders away? Vaults like this allow you to not have that cryptographic material on the server. Now, this doesn't solve all problems. As you mentioned, when you have breached server security you have access to everything. But, there are other problematic situations that this helps solve. Now you don't have to provide secrets when you are building images and you will not have secrets on the filesystem when somebody only gains partial access that lets him only read files. This also lets you see who and when is accessing secrets and lets you manage secrets (think how you are going to replace database password?)

foobiekr 8y ago

You put the service account token on the filesystem and then have it removed from the filesystem after loading it. then it lives inside a process, where it is still recoverable by a determined adversary who attempts to obtain root and can inspect process memory, but you can further protect that by encrypting it in memory with an ephemeral key (which also should not be at a predictable location).

csours 8y ago

Very simply, it removes the possibility of keys showing up in config in source control or in server backups.

nineteen999 8y ago
I like Ansible's approach to this better. We keep all our secrets in ansible-vault and have a password to unlock it for those "who need to know". Sure, this means provisioning VM's requires human interaction to unlock the vault (once) during a kickstart. So what. I'd rather do that than have plaintext or even pre-hashed passwords lying around in source control. As others have pointed out, Hashicorp Vault seems like a good solution to the problems of another Hashicorp product ... Terraform, which used to keep passwords in its state files (not sure if it still does).
- athenot 8y ago
  You can also configure Ansible to lookup to Hashicorp Vault on the fly. https://docs.ansible.com/ansible/2.5/plugins/lookup/hashi_vault.html https://docs.ansible.com/ansible/2.5/plugins/lookup/hashi_va...
  - nineteen999 8y ago
    Unfortunately it wouldn't work for our network even if I wanted to use it, since our systems aren't allowed to connect outbound to the Internet (for good reason). I suppose I could set up a Vault server locally (?), but it's complete overkill for a problem we already solved in a simpler way. Ansible is gradually adding modules for about everything. It's come along way since I started using it back in the 1.8 or 1.9 days.
    - jyounker 8y ago
      Vault is usually hosted in-house. Vault offers features such as short-term secrets which are unique to each client. Vault itself manages creation and destruction of the credentials on the server, allowing it to enforce credential lifetimes

tetha 8y ago

You're looking at a bad example. Good backup encryption generates a local AES key on a ramdisk, encrypts the backup with that, decrypts the AES key with a public key and stores the result. This makes it cryptographically hard to access the backups. The host never has access to the private key to decrypt the AES key to encrypt the backup into a usable form. There's no vault involved there because public keys aren't security relevant. On the other hand, we're running a lot of mutual TLS authentication via CAs in vault. Vault has the private key of these CAs, and authenticated hosts can request signed client certificates for these CA chains. In this case, vault enforces certificate parameters, TTLs, CLR and other things. You could keep requesting certs, but once I revoke the vault authentication you stole, you're done once the certificate expires within 3 days. And on top of that - yes, it's possible to escalate to vault using local credentials for vault. However, that's a very different threat level. Botnets won't abuse your vault instance. Heck, script kiddies wouldn't know how to abuse your vault instance after getting shell access. If someone can RCE into a box, and find your vault integration, and abuse your vault integration, you're in for a ride and usually, vault won't be your problem at this point.

nightbrawler 8y ago

We use Vault in our environment and deployments... Vault has a concept of using tokens for authentication to pull secrets. You can assign max number of uses, max TTL and other parameters when you generate them. For our deployments, we generate a single token with a max # of uses that matches the target server count for our deploy and also a very short TTL of 5 seconds. Our code gets pushed and the token is passed to each server (in an env var) during that time and a command is passed to refresh environment vars with secrets from Vault. So if a token gets compromised it's very likely to be used up and/or expired.

shawnz 8y ago

In addition to what others have said, having a secrets management system like this makes re-keying easier in the event of a compromise. If one app is compromised, you can easily update the DB keys without having to recompile/redeploy every app that uses the DB. You would still have to redeploy the compromised app to change its keys for accessing the secrets management service, though.

atmosx 8y ago

Vault provides telemetry. The secret request patterns for each service are available. Depending on complexity,anomaly detection can be build on top.

zxcvbn4038 8y ago

The real utility of these products is in generating dynamic short lived credentials. The longer a set of credentials exist the greater chance they end up written down or widely distributednor going home with a former employee. With something like Vault your credentials have a lifespan of days, hours, minutes, whatever the shortest timespan your comfortable with is. That limits your attack surface greatly. If your doing a break glass procedure you can be assured that credentials are revoked after a reasonable amount of time.