4 ms·

This is a chance to reiterate best practices: Credentials and other secrets, like API keys, should never be hard-coded in the source code repo. Use some sort

This is a chance to reiterate best practices: Credentials and other secrets, like API keys, should never be hard-coded in the source code repo. Use some sort of secrets management or configuration for that kind of stuff.

ryanSrich 4y ago
In this case, can we confirm that Heroku environment variables were not accessed? Because if they were, even not storing secrets in the source code wouldn’t have prevented a breach. If Heroku could confirm environment variables were safe I’d have a much better sleep tonight.
- ameliaquining 4y ago
  The announcement claims that, as far as they know, the attacker can't access your Heroku account, only your GitHub repo (because they only got the GitHub-issued token). So this would imply that your environment variables are safe. That being said, it sounds like they don't yet know how the attacker managed to do what they did, which means they can't rule out the possibility that more data was stolen than is currently known. But it sounds like probably not.
- drusepth 4y ago
  >The compromised tokens could provide the threat actor access to customer GitHub repos, but not customer Heroku accounts. With the access to customer OAuth tokens, the threat actor may have read and write access to customer GitHub repositories connected to Heroku. Based on the above information, my assumption is that the attacker gained access to code repos hosted on GitHub, but not access to live dynos or the Heroku dashboard (that happily shows ENV vars). We'll see if this information updates as the investigation progresses, though. However, given 1) write access to a github repo, and 2) auto-deployments from github to production dynos (if enabled), an attacker could exfiltrate env vars (among many other nasty things). However, this would trigger events in your app's activity log (new commits + deploys) and should be quick to verify that it didn't happen.
maccard 4y ago
at work thats fine, we use vault or secrets manager, but neither of those are really suitable for self hosting for toy apps. What do you tell someone using github's free tier and aws/gcp's free tier?
- samcheng 4y ago
  Most Heroku users are using environment variables for their secrets.
- nijave 4y ago
  You can store them in the repo and encrypt them with Mozilla sops. You can use a cloud managed key like AWS KMS or a GPG key. If you manage your own key, you can store it in a password manager or use a USB hardware key to store it You could also use object/blob storage or your local filesystem to store a config file and optionally apply encryption to that
  - maccard 4y ago
    You've just moved the problem from storing the secret to storing the key in that case though. For AWS KMS, you're now paying [0] for the key storage, and if you use a GPG key, you still need to get it into the application somehow. [0] https://aws.amazon.com/kms/pricing/ https://aws.amazon.com/kms/pricing/
    - nijave 4y ago
      If you're using a GPG key, you don't necessarily need it to leave your PC. You just need to "re-deploy" the secrets manually when they change from a place that has access to the key

ryanSrich 4y ago

In this case, can we confirm that Heroku environment variables were not accessed? Because if they were, even not storing secrets in the source code wouldn’t have prevented a breach. If Heroku could confirm environment variables were safe I’d have a much better sleep tonight.

ameliaquining 4y ago
The announcement claims that, as far as they know, the attacker can't access your Heroku account, only your GitHub repo (because they only got the GitHub-issued token). So this would imply that your environment variables are safe. That being said, it sounds like they don't yet know how the attacker managed to do what they did, which means they can't rule out the possibility that more data was stolen than is currently known. But it sounds like probably not.
drusepth 4y ago
>The compromised tokens could provide the threat actor access to customer GitHub repos, but not customer Heroku accounts. With the access to customer OAuth tokens, the threat actor may have read and write access to customer GitHub repositories connected to Heroku. Based on the above information, my assumption is that the attacker gained access to code repos hosted on GitHub, but not access to live dynos or the Heroku dashboard (that happily shows ENV vars). We'll see if this information updates as the investigation progresses, though. However, given 1) write access to a github repo, and 2) auto-deployments from github to production dynos (if enabled), an attacker could exfiltrate env vars (among many other nasty things). However, this would trigger events in your app's activity log (new commits + deploys) and should be quick to verify that it didn't happen.

maccard 4y ago

at work thats fine, we use vault or secrets manager, but neither of those are really suitable for self hosting for toy apps. What do you tell someone using github's free tier and aws/gcp's free tier?

samcheng 4y ago
Most Heroku users are using environment variables for their secrets.
nijave 4y ago
You can store them in the repo and encrypt them with Mozilla sops. You can use a cloud managed key like AWS KMS or a GPG key. If you manage your own key, you can store it in a password manager or use a USB hardware key to store it You could also use object/blob storage or your local filesystem to store a config file and optionally apply encryption to that
- maccard 4y ago
  You've just moved the problem from storing the secret to storing the key in that case though. For AWS KMS, you're now paying [0] for the key storage, and if you use a GPG key, you still need to get it into the application somehow. [0] https://aws.amazon.com/kms/pricing/ https://aws.amazon.com/kms/pricing/
  - nijave 4y ago
    If you're using a GPG key, you don't necessarily need it to leave your PC. You just need to "re-deploy" the secrets manually when they change from a place that has access to the key