6 ms·

Do not use secrets in environment variables

79 points by dangtony98 2y ago

jampekka 2y ago
Security through complexity.
Circlecrypto2 2y ago
I actually liked this article. A great explainer in why environment variables are a terrible idea. Nike actually open-sourced their Keystore solution if anyone is curious. It was called cerberus.
- moritonal 2y ago
  TIL Nike (the shoe company) has a surprisingly intense OSS presence. https://github.com/orgs/Nike-Inc/repositories https://github.com/orgs/Nike-Inc/repositories
  - adhamsalama 2y ago
    > hal provides an AWS Lambda Custom Runtime environment for your Haskell applications. Didn't expect that from Nike.
  - zemo 2y ago
    > Nike (the shoe company) as opposed to Nike, the Ancient Greek goddess of Victory, who contributes essentially nothing back to the OSS community
    - bigiain 2y ago
      We need to immediately block Nike (the Goddess) from wordpress.org until she commits to paying 8% of her gross revenue to Matt's personal bank account.
  - danillonunes 2y ago
    Oh you work in tech and never heard of the strong influence of the FAANG (Fila, Adidas, ASICS, Nike, Gucci)?
    - moritonal 2y ago
      Okay, I really thought you were messing with me, but both https://github.com/ADIDAS https://github.com/ADIDAS and https://gucci-kering-group.opendatasoft.com/pages/home/ https://gucci-kering-group.opendatasoft.com/pages/home/ do have project pages!
liveoneggs 2y ago
this is mostly about not using .env files
- diggan 2y ago
  I'm not sure if I understood something wrong, but .env files are for development, not production. It never crossed my mind to use .env files for production deployments, but skimming the blog post, it seems like people do ship a .env file to their production environment?
  - stackskipton 2y ago
    SRE here, most of time when that happens, it's people leaving .env in git where it's not ignored and build process just does COPY * * to runtime environment. EDIT: Or it's just small time developers who don't care about security and ship whatever works.
    - deleted 2y ago
      [deleted]
    - hinkley 2y ago
      Hidden files get checked/built in all the time. Too many people don’t think to look for them.
      - cwbriscoe 2y ago
        That is why I like "Allow Listing" in my .gitignore file. Much harder to add stuff by accident. # Ignore everything * # But not these files... !/.gitignore !*.go !go.sum !go.mod
        hinkley 2y ago
        I do the same. But as I do it my mental image of myself is of someone holding their nose and typing one handed. It’s the least shit solution, but it’s definitely a shit solution. Npm has another way to default exclude but I don’t think git does, and if docker got one I don’t recall what it is.
spankalee 2y ago
In what world does this work? $ curl http://your-website.com/public/../../../../proc/12345/environ If your server is serving up your whole filesystem, you likely have a lot of big problems.
- richwater 2y ago
  You'd be surprised...
- mlaux 2y ago
  Typically seen more often as something like: http://example.com/viewPost.php?post=../../../whatever (where the server side code has the bug, not the web server configuration itself)
  - hinkley 2y ago
    We got access logs like this all the time. SaaS app with separate domain per customer so we got a lot of probes like this. Famously IIS had some bugs like this. Not surprised that PHP has problems. What a clown car.
    - mschuster91 2y ago
      > Famously IIS had some bugs like this. Not surprised that PHP has problems. What a clown car. This is a file inclusion vulnerability. And that's something that is _very_ easily done in any language which people use to do stuff that loads files from the file system. Image resizers are a pretty common exploit path.
    - Dylan16807 2y ago
      That is not an example of a php problem. There is no "post=filename" feature in php; it's running webdev code. They only used .php because it's a very common language.
- PaulDavisThe1st 2y ago
  and even if your server can reach it, here (Debian 11), all of /proc/PID/environ is marked 0400, root:root owned.
cartofupai 2y ago
“Even if exposed or leaked, it is one secret to rotate rather than all of your secrets, scattered across all of your services and their environment variables.” I don’t think this is true. You’d rotate all secrets in the store, as they could be accessed/compromised.
- Groxx 2y ago
  You'd hopefully have access records to show which ones were accessed... but yea, all accessed secrets need to be rotated, not just the gateway-secret. And without that kind of record you need to assume "all".
- neilk 2y ago
  That’s what 1Password means, right? If your credentials to 1Password leaks all you have to do is change that one password. Problem solved! /s
splintfrog89 2y ago
External "Secrets management services" are among the most attractive hacking targets. It is beyond me why you would have full trust in those.
- candiddevmike 2y ago
  Because the alternative of scattered, DIY secret stores is much worse.
  - g8oz 2y ago
    OTH they limit the blast radius.
  - acdha 2y ago
    If your DIY secret store is an exposed network service, maybe. If your secret store is a set of conventions which keeps access confined to the application environment, no. Things like Ansible Vault, AWS/Azure/GCP/etc. secrets using role-based, etc. have the nice property that they are isolated from unrelated apps so an attacker can’t breach one thing and move laterally across all of your applications. You have to protect that core infrastructure anyway so there’s an argument for not doing so more times than necessary.
- nemo 2y ago
  Secrets in env variables are a highly attractive hacking target, they leak easily and can spill the keys to the kingdom.
jerf 2y ago
Here, let me boil it all down for you. Basically, you can determine if it's safe to store secrets in a given place by feeding it to this Python function, which will return True if it's safe and False if it is not: def canIStoreMySecretsHere(location): return False Basically, for any location you might store a secret, a hacker might get access to it. Therefore, it is not safe there. You might think I'm being sarcastic, but... perhaps less than you'd think. It has often seemed to me that secret management is a game of temporal arbitrage, where you stick them in some new sort of place and just pretend that that new place must be secure, until you realize some time later it is not, and then you stick it in a new place, a new "secrets manager" that is safe, until that gets popped, then you stick it somewhere else.... (Note this is about symmetric secrets, and things like passwords. Asymmetric things admit more interesting possibilities of bundling some computation with the storage with things like secure enclaves. One can debate the physical security of a secure enclave, but assuming its software is correctly implemented, a secret store where there simply is no API in theory or in practice to extract the secret back out is an actual improvement in secret storage that I am not sarcastic about.)
- xp84 2y ago
  I guess you're being downvoted for sarcasm, but I agree with you that ultimately the "solution" of "just use some secrets manager from the cloud vendor" is 100% a shell game. The arguments here are mostly the same logical fallacy that makes most DRM a joke: In order for the application to function (the end user to view the movie) they must be able to use the secret (the decryption key(s) of the movie). You can play as many games of indirection as you want, but it's all basically security by obscurity. If your application is required to fetch its secrets from AWS or Google, then someone with the type of access needed to dump the environment vars (i.e. root access on the box) can likely also modify your application code, for instance, to dump out those fetched secrets to a file.)
- hinkley 2y ago
  One of the big uses of hardware security modules was allowing web servers to reboot without the private keys being accessible through the filesystem. But they keep people from walking off with private keys, they don’t stop people from borrowing them. HSMs are great for SSL keys but not so great for code or cert signing.
dolni 2y ago
This article has about as much insight as I would expect from a "nodejs-security.com" article. This article spends a good deal of time conflating two things: putting stuff in .env and using environment variables. The application-parsed .env file is one of the most poorly thought-through ideas that has taken hold in modern application development. It takes something you can do in literally a couple lines of shell (as a container entrypoint) and adds a bunch of complexity for something that is actually just worse. In local dev scenarios, app-parsed .env files suck because you often end up with some kind of dev-specific secret that you don't want committed to the app repo. In my experience this means developers figure out how to pass a .env file around. If you use an actual shell instead, the local-dev .env can shell out to something like the AWS CLI to get secrets from parameter store. Or you could grab them from Hashicorp Vault if you run that. And because a shell fetches it at run time, secret updates are seamless and properly access-controlled in one spot. In proper deployment scenarios, .env sucks because your deployment system (container orchestrator, Lambda, etc) will need to set those values appropriately for their current environment anyway. And by having a .env file the app loads, now you have two places for configuration. Applications simply should not have any involvement in setting values for their own environment variables. They are typically used for core infrastructure-level configuration. The source of truth for this is probably going to be available via something like Terraform. So the application should ultimately inherit they configuration through Terraform. Additionally, this article is simply wrong on environment variables being readable by any user on a Linux system. On Linux, a process's environment can be read by the superuser and the user who owns the process. That's it.
- mhitza 2y ago
  You're going from something simple straight to Terraform and secret storage systems like Vault (KMS and similar). While I'm not the biggest fan of where we are right now, it's better than the complexity of integrating with a secret engines. What I really think is ideal is to dump individual secrets into files and only load the decryption key into memory. That way at runtime secrets can be read and decrypted without polluting ENV and avoiding them from lingering around. Another integration, that would be great if it existed, would be if more web frameworks (the perspective I'm seeing this problem from) would integrate with the Linux memfd_secret api. Which would work great with implementations such as php-fcgi, but no so much with greeanthreaded systems.
pphysch 2y ago
systemd 247+ now discourages using env vars for secrets, because there are the LoadCredential and related options that are much more secure, with better isolation and encryption features. https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html https://www.freedesktop.org/software/systemd/man/latest/syst...
- candiddevmike 2y ago
  I would argue that's a systemd quirk due to D-Bus design. Environment variables for processes in Linux for the most part cannot be read by processes of non-root users, so out of the box they are "secure" for the user starting the process.
  - NewJazz 2y ago
    They are also inherited by child processes. Though, so is the mount namespace.
  - pphysch 2y ago
    Right, but dumping envs is a common debugging and attack vector.
- asqueella 2y ago
  Thanks, wouldn’t think of environment being shared via DBus: > Note that environment variables are not suitable for passing secrets (such as passwords, key material, …) to service processes. Environment variables set for a unit are exposed to unprivileged clients via D-Bus IPC, and generally not understood as being data that requires protection. > Moreover, environment variables are propagated down the process tree, including across security boundaries (such as setuid/setgid executables), and hence might leak to processes that should not have access to the secret data.
mvdtnz 2y ago
Hang on a second. One of the reasons is you might "do this", where "this" is rendering your environment variables into your HTML. First of all, just don't do that. Secondly, What's stopping you from doing the same thing with a secret in a proper managed secret repository?
jakewins 2y ago
So don’t provide secrets to your service via environment variable, it can leak? Instead, expose all your secrets via a public API with IP filtering, then give the credentials to this service to your app - as an environment variable - and voila! This just seems like increasing complexity in a part of your system that should be as simple and non-dynamic as humanly possible, for the upside of.. much larger attack surface? We inject secrets, as env vars, in each Deployment. No runtime access to any secrets store, gives the ability to generate new secrets for each deployment, and minimal complexity for such a critical aspect of the system.
- hinkley 2y ago
  Our (very) old system used enciphered secrets and an init script that got the key into a file and then unset the environment variable before the app started. But we never had a good system for sandboxes, which meant another way the local and deployment differed. They moved to a secret store instead. I don’t know if that’s better.
politelemon 2y ago
> To call out a practical example: 1Password. Of all the examples that could have been 'called out', this is the least practical one. Jumping from the problem statement straight to hosting with a third party provider completely ignores the huge risk that comes with it. Using environment variables is risky so just give your secrets to some third party... which then provides environment variables anyway. This entire section ought to be dropped, it almost reads like a sponsored bit and there are much better and more widely used solutions used such as sops and vault.
- theozero 2y ago
  To be fair, 1password is designed so that they cannot actually access your secrets, and it's likely many of your sensitive secrets are already stored there for use by humans.
- favorited 2y ago
  1Password is a reasonable example because they're a big player in the enterprise secrets management market, so your company might already have it available for you to use. They didn't get a $6.7 billion valuation by managing end-user passwords in a world where most people have a single password they use for everything.
erulabs 2y ago
Aside from the .env points, which, fair enough, don't use "environment variables in a file", this always boils down to one actual security concern: Are you okay with every piece of code in your codebase having quick and easy access to all secrets? The answer _feels_ like it should be no: zero-trust-by-default, etc. but you're fooling yourself - a compromised dependency isn't _just_ going to look at process.env. It's going to be installing a backdoor and having an agent login and poke around. It's going to be netcatting for 3306 and finding out where the credentials are eventually. Security thru obscurity is no security at all. Last thought, Kubernetes' `envFrom` is such a salve of simplicity in this day and age.
mbrumlow 2y ago
Don't use secrets in environment variables, but use this secret in environment variables, but this one gives you access to all secrets. Things like vault which was suggested still requires you to pass the vault token in to your app somehow. And even then if your application does not have direct vault support you will still be using vault to supply secrets via environment variables, its even the recommended way with Nomad and their template system. I really dislike these sort of articles. Because it has a catchy phrase "Do not use secrets in environment variables", and that is all that will be remembered. And next thing you know you will be at a company submitting a PR and some guy will say "Do not use secrets in environment variables", and then advise you to pass them as arguments on the command line (this happened to me). Environment variables are today, the most safe way to pass secrets to a program. .env files ARE NOT Environment variables, they are files. A better title and write up would be "Do not store secrets in file". Once you do that all the weird problems described, with exception of printing them as logs, go away. Then you need a new article "Do not print secrets in your program". But that is all moot, because you should already be filtering out secrets by configuring your local log system to do so. I my self write wrappers and log systems that handle filtering out secrets from logs within the same context of the application I run. Its super simple, it s fast (if you know what a trie is) and you can worry free print secrets. Edit: This article in fact has more damaging impact to security as a whole, mostly because of the conflation and 99/1 on problem/solution. This entire domain will now be blocked on all networks I have control over.
- thayne 2y ago
  > Things like vault which was suggested still requires you to pass the vault token in to your app somehow. And even then if your application does not have direct vault support you will still be using vault to supply secrets via environment variables, its even the recommended way with Nomad and their template system. Indeed. But ideally the "somehow" that you pass that in is not an environment variable. Maybe something like a secure in-memory file that only your container/systemd service/whatever has access to that's injected by something like kubernetes/docker secrets, or systemd-creds, etc.
  - mbrumlow 2y ago
rihards__ 2y ago
I believe this sort of sums it up: “To begin with, the hint lies in the title of this section: secrets management. Environment variables are hardly managed unless you explicitly use an integration or an orchestrator like Kubernetes to automatically inject environment variable configuration.” It has nothing to do with environment variables… It is just a ephermal way how to inject variables into the process, if you do it right…
madamelic 2y ago
The problem isn't the tech, it's the people. If you allow these mistakes to be possible, they are inevitable. If you take basic precautions, you'll probably be fine. I'd rather take a well-curated and trimmed down .env over a poorly-configured secrets manager that gives away the entire farm when the single secret leaks. Security isn't a single thing nor bolstered by switching a single method of how you store your secrets. The problem is not taking precautions to prevent leaks from happening, not how you are managing your secrets. If your threat model begins or is imminently "when the attacker is logged in as root", just post your stuff on a public bucket to get it over with.
deleted 2y ago
[deleted]
necovek 2y ago
I was surprised to see so much comments here upon reading the article: the article is full of bad takes, and all the commenters seem to agree. This is simply bad advice, let's get this off the frontpage.
knowsuchagency 2y ago
Environment variables are fine just don’t bake them into your image. K8s and docker support secrets that are just environment variables from the applications’ business logic. You can trivially read them from a secret and populate the environment at runtime, also.
jfyalkdfwuasdf 2y ago
This is why I beg everyone to adopt workload identity federation.
2000swebgeek 2y ago
ENV variables having the names with PASSWORD or SECRET should be ignored by logging and monitoring systems. Most of the web has been built on trust of following conventions. common secrets used on server side - `JWT_SECRET`, `DATABASE_PASSWORD`, `PGPASSWORD`, `AWS_SECRET_TOKEN` etc., Being a long time developer, this breaks the standard of backend apps which mostly uses 12 Factor App[1]. This approach introduces a new dependency for fetching secrets. I see all new open-source projects using "paid" or "hosted" solutions. It is no longer easy/simple to host a full open-source app without external dependencies. (I understand -- things are getting complicated with s3 for storage etc.,). [1] https://12factor.net/ https://12factor.net/
elderlybanana 2y ago
In my experience, juniors don't know how to do a secure setup, and busy seniors are often willing to cut corners to complete business-critical tasks. Application secrets management needs a better default setup. The standard for application auth should be more similar to an IAM system. The dev community needs to find a better default than .env files for secrets. While there are plenty of alternatives, they generally all require knowledge of some third party system, which most people, for many reasons do not have the time or interest to learn, and some 3rd party secret to unlock the rest. We need better default abstractions around secrets management. The authentication step to fetch secrets should be pushed to something ephemeral, probably biometrics. Ideally, devs should almost never interact with secrets in any way. They should use secure and convenient MFA methods to authN/Z their access to services, and secrets management happens out of sight. And this should all happen automatically with default tooling. It is fairly easy to authenticate between services without secrets in the context of a single platform like AWS using IAM policies and roles, but I think we need to solve the more general case for secrets management abstraction across platforms and services. OSs, browsers, and dev tooling are becoming more mature with respect to auth methods. Secrets management should be mostly the domain of a select group of people, like any number of other complex computer systems details.
gerardnico 2y ago
I made a direnv extension for that purposes. It loads env files and call hashicorp vault if the value is a secret. I find it pretty neat to have an env file that describes all environments variables. https://github.com/gerardnico/direnv-ext https://github.com/gerardnico/direnv-ext
amingilani 2y ago
This may be the first time I’ve looked at an article and realized that it is AI slop.

79 points by dangtony98 2y ago

Security through complexity.

I actually liked this article. A great explainer in why environment variables are a terrible idea. Nike actually open-sourced their Keystore solution if anyone is curious. It was called cerberus.

moritonal 2y ago
TIL Nike (the shoe company) has a surprisingly intense OSS presence. https://github.com/orgs/Nike-Inc/repositories https://github.com/orgs/Nike-Inc/repositories
- adhamsalama 2y ago
  > hal provides an AWS Lambda Custom Runtime environment for your Haskell applications. Didn't expect that from Nike.
- zemo 2y ago
  > Nike (the shoe company) as opposed to Nike, the Ancient Greek goddess of Victory, who contributes essentially nothing back to the OSS community
  - bigiain 2y ago
    We need to immediately block Nike (the Goddess) from wordpress.org until she commits to paying 8% of her gross revenue to Matt's personal bank account.
- danillonunes 2y ago
  Oh you work in tech and never heard of the strong influence of the FAANG (Fila, Adidas, ASICS, Nike, Gucci)?
  - moritonal 2y ago
    Okay, I really thought you were messing with me, but both https://github.com/ADIDAS https://github.com/ADIDAS and https://gucci-kering-group.opendatasoft.com/pages/home/ https://gucci-kering-group.opendatasoft.com/pages/home/ do have project pages!

liveoneggs 2y ago

this is mostly about not using .env files

diggan 2y ago
I'm not sure if I understood something wrong, but .env files are for development, not production. It never crossed my mind to use .env files for production deployments, but skimming the blog post, it seems like people do ship a .env file to their production environment?
- stackskipton 2y ago
  SRE here, most of time when that happens, it's people leaving .env in git where it's not ignored and build process just does COPY * * to runtime environment. EDIT: Or it's just small time developers who don't care about security and ship whatever works.
  - deleted 2y ago
    [deleted]
  - hinkley 2y ago
    Hidden files get checked/built in all the time. Too many people don’t think to look for them.
    - cwbriscoe 2y ago
      That is why I like "Allow Listing" in my .gitignore file. Much harder to add stuff by accident. # Ignore everything * # But not these files... !/.gitignore !*.go !go.sum !go.mod
      - hinkley 2y ago
        I do the same. But as I do it my mental image of myself is of someone holding their nose and typing one handed. It’s the least shit solution, but it’s definitely a shit solution. Npm has another way to default exclude but I don’t think git does, and if docker got one I don’t recall what it is.

spankalee 2y ago

In what world does this work? $ curl http://your-website.com/public/../../../../proc/12345/environ If your server is serving up your whole filesystem, you likely have a lot of big problems.

richwater 2y ago
You'd be surprised...
mlaux 2y ago
Typically seen more often as something like: http://example.com/viewPost.php?post=../../../whatever (where the server side code has the bug, not the web server configuration itself)
- hinkley 2y ago
  We got access logs like this all the time. SaaS app with separate domain per customer so we got a lot of probes like this. Famously IIS had some bugs like this. Not surprised that PHP has problems. What a clown car.
  - mschuster91 2y ago
    > Famously IIS had some bugs like this. Not surprised that PHP has problems. What a clown car. This is a file inclusion vulnerability. And that's something that is _very_ easily done in any language which people use to do stuff that loads files from the file system. Image resizers are a pretty common exploit path.
  - Dylan16807 2y ago
    That is not an example of a php problem. There is no "post=filename" feature in php; it's running webdev code. They only used .php because it's a very common language.
PaulDavisThe1st 2y ago
and even if your server can reach it, here (Debian 11), all of /proc/PID/environ is marked 0400, root:root owned.

cartofupai 2y ago

“Even if exposed or leaked, it is one secret to rotate rather than all of your secrets, scattered across all of your services and their environment variables.” I don’t think this is true. You’d rotate all secrets in the store, as they could be accessed/compromised.

Groxx 2y ago
You'd hopefully have access records to show which ones were accessed... but yea, all accessed secrets need to be rotated, not just the gateway-secret. And without that kind of record you need to assume "all".
neilk 2y ago
That’s what 1Password means, right? If your credentials to 1Password leaks all you have to do is change that one password. Problem solved! /s

splintfrog89 2y ago

External "Secrets management services" are among the most attractive hacking targets. It is beyond me why you would have full trust in those.

candiddevmike 2y ago
Because the alternative of scattered, DIY secret stores is much worse.
- g8oz 2y ago
  OTH they limit the blast radius.
- acdha 2y ago
  If your DIY secret store is an exposed network service, maybe. If your secret store is a set of conventions which keeps access confined to the application environment, no. Things like Ansible Vault, AWS/Azure/GCP/etc. secrets using role-based, etc. have the nice property that they are isolated from unrelated apps so an attacker can’t breach one thing and move laterally across all of your applications. You have to protect that core infrastructure anyway so there’s an argument for not doing so more times than necessary.
nemo 2y ago
Secrets in env variables are a highly attractive hacking target, they leak easily and can spill the keys to the kingdom.

jerf 2y ago

Here, let me boil it all down for you. Basically, you can determine if it's safe to store secrets in a given place by feeding it to this Python function, which will return True if it's safe and False if it is not: def canIStoreMySecretsHere(location): return False Basically, for any location you might store a secret, a hacker might get access to it. Therefore, it is not safe there. You might think I'm being sarcastic, but... perhaps less than you'd think. It has often seemed to me that secret management is a game of temporal arbitrage, where you stick them in some new sort of place and just pretend that that new place must be secure, until you realize some time later it is not, and then you stick it in a new place, a new "secrets manager" that is safe, until that gets popped, then you stick it somewhere else.... (Note this is about symmetric secrets, and things like passwords. Asymmetric things admit more interesting possibilities of bundling some computation with the storage with things like secure enclaves. One can debate the physical security of a secure enclave, but assuming its software is correctly implemented, a secret store where there simply is no API in theory or in practice to extract the secret back out is an actual improvement in secret storage that I am not sarcastic about.)

xp84 2y ago
I guess you're being downvoted for sarcasm, but I agree with you that ultimately the "solution" of "just use some secrets manager from the cloud vendor" is 100% a shell game. The arguments here are mostly the same logical fallacy that makes most DRM a joke: In order for the application to function (the end user to view the movie) they must be able to use the secret (the decryption key(s) of the movie). You can play as many games of indirection as you want, but it's all basically security by obscurity. If your application is required to fetch its secrets from AWS or Google, then someone with the type of access needed to dump the environment vars (i.e. root access on the box) can likely also modify your application code, for instance, to dump out those fetched secrets to a file.)
hinkley 2y ago
One of the big uses of hardware security modules was allowing web servers to reboot without the private keys being accessible through the filesystem. But they keep people from walking off with private keys, they don’t stop people from borrowing them. HSMs are great for SSL keys but not so great for code or cert signing.

dolni 2y ago

This article has about as much insight as I would expect from a "nodejs-security.com" article. This article spends a good deal of time conflating two things: putting stuff in .env and using environment variables. The application-parsed .env file is one of the most poorly thought-through ideas that has taken hold in modern application development. It takes something you can do in literally a couple lines of shell (as a container entrypoint) and adds a bunch of complexity for something that is actually just worse. In local dev scenarios, app-parsed .env files suck because you often end up with some kind of dev-specific secret that you don't want committed to the app repo. In my experience this means developers figure out how to pass a .env file around. If you use an actual shell instead, the local-dev .env can shell out to something like the AWS CLI to get secrets from parameter store. Or you could grab them from Hashicorp Vault if you run that. And because a shell fetches it at run time, secret updates are seamless and properly access-controlled in one spot. In proper deployment scenarios, .env sucks because your deployment system (container orchestrator, Lambda, etc) will need to set those values appropriately for their current environment anyway. And by having a .env file the app loads, now you have two places for configuration. Applications simply should not have any involvement in setting values for their own environment variables. They are typically used for core infrastructure-level configuration. The source of truth for this is probably going to be available via something like Terraform. So the application should ultimately inherit they configuration through Terraform. Additionally, this article is simply wrong on environment variables being readable by any user on a Linux system. On Linux, a process's environment can be read by the superuser and the user who owns the process. That's it.

mhitza 2y ago
You're going from something simple straight to Terraform and secret storage systems like Vault (KMS and similar). While I'm not the biggest fan of where we are right now, it's better than the complexity of integrating with a secret engines. What I really think is ideal is to dump individual secrets into files and only load the decryption key into memory. That way at runtime secrets can be read and decrypted without polluting ENV and avoiding them from lingering around. Another integration, that would be great if it existed, would be if more web frameworks (the perspective I'm seeing this problem from) would integrate with the Linux memfd_secret api. Which would work great with implementations such as php-fcgi, but no so much with greeanthreaded systems.

pphysch 2y ago

systemd 247+ now discourages using env vars for secrets, because there are the LoadCredential and related options that are much more secure, with better isolation and encryption features. https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html https://www.freedesktop.org/software/systemd/man/latest/syst...

candiddevmike 2y ago
I would argue that's a systemd quirk due to D-Bus design. Environment variables for processes in Linux for the most part cannot be read by processes of non-root users, so out of the box they are "secure" for the user starting the process.
- NewJazz 2y ago
  They are also inherited by child processes. Though, so is the mount namespace.
- pphysch 2y ago
  Right, but dumping envs is a common debugging and attack vector.
asqueella 2y ago
Thanks, wouldn’t think of environment being shared via DBus: > Note that environment variables are not suitable for passing secrets (such as passwords, key material, …) to service processes. Environment variables set for a unit are exposed to unprivileged clients via D-Bus IPC, and generally not understood as being data that requires protection. > Moreover, environment variables are propagated down the process tree, including across security boundaries (such as setuid/setgid executables), and hence might leak to processes that should not have access to the secret data.

mvdtnz 2y ago

Hang on a second. One of the reasons is you might "do this", where "this" is rendering your environment variables into your HTML. First of all, just don't do that. Secondly, What's stopping you from doing the same thing with a secret in a proper managed secret repository?

jakewins 2y ago

So don’t provide secrets to your service via environment variable, it can leak? Instead, expose all your secrets via a public API with IP filtering, then give the credentials to this service to your app - as an environment variable - and voila! This just seems like increasing complexity in a part of your system that should be as simple and non-dynamic as humanly possible, for the upside of.. much larger attack surface? We inject secrets, as env vars, in each Deployment. No runtime access to any secrets store, gives the ability to generate new secrets for each deployment, and minimal complexity for such a critical aspect of the system.

hinkley 2y ago
Our (very) old system used enciphered secrets and an init script that got the key into a file and then unset the environment variable before the app started. But we never had a good system for sandboxes, which meant another way the local and deployment differed. They moved to a secret store instead. I don’t know if that’s better.

politelemon 2y ago

> To call out a practical example: 1Password. Of all the examples that could have been 'called out', this is the least practical one. Jumping from the problem statement straight to hosting with a third party provider completely ignores the huge risk that comes with it. Using environment variables is risky so just give your secrets to some third party... which then provides environment variables anyway. This entire section ought to be dropped, it almost reads like a sponsored bit and there are much better and more widely used solutions used such as sops and vault.

theozero 2y ago
To be fair, 1password is designed so that they cannot actually access your secrets, and it's likely many of your sensitive secrets are already stored there for use by humans.
favorited 2y ago
1Password is a reasonable example because they're a big player in the enterprise secrets management market, so your company might already have it available for you to use. They didn't get a $6.7 billion valuation by managing end-user passwords in a world where most people have a single password they use for everything.

erulabs 2y ago

Aside from the .env points, which, fair enough, don't use "environment variables in a file", this always boils down to one actual security concern: Are you okay with every piece of code in your codebase having quick and easy access to all secrets? The answer _feels_ like it should be no: zero-trust-by-default, etc. but you're fooling yourself - a compromised dependency isn't _just_ going to look at process.env. It's going to be installing a backdoor and having an agent login and poke around. It's going to be netcatting for 3306 and finding out where the credentials are eventually. Security thru obscurity is no security at all. Last thought, Kubernetes' `envFrom` is such a salve of simplicity in this day and age.

mbrumlow 2y ago

Don't use secrets in environment variables, but use this secret in environment variables, but this one gives you access to all secrets. Things like vault which was suggested still requires you to pass the vault token in to your app somehow. And even then if your application does not have direct vault support you will still be using vault to supply secrets via environment variables, its even the recommended way with Nomad and their template system. I really dislike these sort of articles. Because it has a catchy phrase "Do not use secrets in environment variables", and that is all that will be remembered. And next thing you know you will be at a company submitting a PR and some guy will say "Do not use secrets in environment variables", and then advise you to pass them as arguments on the command line (this happened to me). Environment variables are today, the most safe way to pass secrets to a program. .env files ARE NOT Environment variables, they are files. A better title and write up would be "Do not store secrets in file". Once you do that all the weird problems described, with exception of printing them as logs, go away. Then you need a new article "Do not print secrets in your program". But that is all moot, because you should already be filtering out secrets by configuring your local log system to do so. I my self write wrappers and log systems that handle filtering out secrets from logs within the same context of the application I run. Its super simple, it s fast (if you know what a trie is) and you can worry free print secrets. Edit: This article in fact has more damaging impact to security as a whole, mostly because of the conflation and 99/1 on problem/solution. This entire domain will now be blocked on all networks I have control over.

thayne 2y ago
> Things like vault which was suggested still requires you to pass the vault token in to your app somehow. And even then if your application does not have direct vault support you will still be using vault to supply secrets via environment variables, its even the recommended way with Nomad and their template system. Indeed. But ideally the "somehow" that you pass that in is not an environment variable. Maybe something like a secure in-memory file that only your container/systemd service/whatever has access to that's injected by something like kubernetes/docker secrets, or systemd-creds, etc.
- mbrumlow 2y ago

rihards__ 2y ago

I believe this sort of sums it up: “To begin with, the hint lies in the title of this section: secrets management. Environment variables are hardly managed unless you explicitly use an integration or an orchestrator like Kubernetes to automatically inject environment variable configuration.” It has nothing to do with environment variables… It is just a ephermal way how to inject variables into the process, if you do it right…

madamelic 2y ago

The problem isn't the tech, it's the people. If you allow these mistakes to be possible, they are inevitable. If you take basic precautions, you'll probably be fine. I'd rather take a well-curated and trimmed down .env over a poorly-configured secrets manager that gives away the entire farm when the single secret leaks. Security isn't a single thing nor bolstered by switching a single method of how you store your secrets. The problem is not taking precautions to prevent leaks from happening, not how you are managing your secrets. If your threat model begins or is imminently "when the attacker is logged in as root", just post your stuff on a public bucket to get it over with.

deleted 2y ago

[deleted]

necovek 2y ago

I was surprised to see so much comments here upon reading the article: the article is full of bad takes, and all the commenters seem to agree. This is simply bad advice, let's get this off the frontpage.

knowsuchagency 2y ago

Environment variables are fine just don’t bake them into your image. K8s and docker support secrets that are just environment variables from the applications’ business logic. You can trivially read them from a secret and populate the environment at runtime, also.

jfyalkdfwuasdf 2y ago

This is why I beg everyone to adopt workload identity federation.

2000swebgeek 2y ago

ENV variables having the names with PASSWORD or SECRET should be ignored by logging and monitoring systems. Most of the web has been built on trust of following conventions. common secrets used on server side - `JWT_SECRET`, `DATABASE_PASSWORD`, `PGPASSWORD`, `AWS_SECRET_TOKEN` etc., Being a long time developer, this breaks the standard of backend apps which mostly uses 12 Factor App[1]. This approach introduces a new dependency for fetching secrets. I see all new open-source projects using "paid" or "hosted" solutions. It is no longer easy/simple to host a full open-source app without external dependencies. (I understand -- things are getting complicated with s3 for storage etc.,). [1] https://12factor.net/ https://12factor.net/

elderlybanana 2y ago

In my experience, juniors don't know how to do a secure setup, and busy seniors are often willing to cut corners to complete business-critical tasks. Application secrets management needs a better default setup. The standard for application auth should be more similar to an IAM system. The dev community needs to find a better default than .env files for secrets. While there are plenty of alternatives, they generally all require knowledge of some third party system, which most people, for many reasons do not have the time or interest to learn, and some 3rd party secret to unlock the rest. We need better default abstractions around secrets management. The authentication step to fetch secrets should be pushed to something ephemeral, probably biometrics. Ideally, devs should almost never interact with secrets in any way. They should use secure and convenient MFA methods to authN/Z their access to services, and secrets management happens out of sight. And this should all happen automatically with default tooling. It is fairly easy to authenticate between services without secrets in the context of a single platform like AWS using IAM policies and roles, but I think we need to solve the more general case for secrets management abstraction across platforms and services. OSs, browsers, and dev tooling are becoming more mature with respect to auth methods. Secrets management should be mostly the domain of a select group of people, like any number of other complex computer systems details.

gerardnico 2y ago

I made a direnv extension for that purposes. It loads env files and call hashicorp vault if the value is a secret. I find it pretty neat to have an env file that describes all environments variables. https://github.com/gerardnico/direnv-ext https://github.com/gerardnico/direnv-ext

amingilani 2y ago

This may be the first time I’ve looked at an article and realized that it is AI slop.