6 ms·
Environment variables are often used to pass secrets around. But, despite its ubiquity, I believe that's a bad practice: - On Linux systems, any user process
by rsyring 1y ago
Environment variables are often used to pass secrets around. But, despite its ubiquity, I believe that's a bad practice:
- On Linux systems, any user process can inspect any other process of that same user for it's environment variables. We can argue about threat model but, especially for a developer's system, there are A LOT of processes running as the same user as the developer.
- IMO, this has become an even more pronounced problem with the popularity of running non-containerized LLM agents in the same user space as the developer's primary OS user. It's a secret exfiltration exploiter's dream.
- Environment variables are usually passed down to other spawned processes instead of being contained to the primary process which is often the only one that needs it.
- Systemd exposes unit environment variables to all system clients through DBUS and warns against using environment variables for secrets[1]. I believe this means non-root users have access to environment variables set for root-only units/services. I could be wrong, I haven't tested it yet. But if this is the case, I bet that's a HUGE surprise to many system admins.
I think ephemeral file sharing between a secret managing process (e.g. 1Password's op cli tool) and the process that needs the secrets (flask, terraform, etc.) is likely the only solution that keeps secrets out of files and also out of environment variables. This is how Systemd's credentials system works. But it's far from widely supported.
Any good solutions for passing secrets around that don't involve environment variables or regular plain text files?
Edit: I think 1Password's op client has a good start in that each new "session" requires my authorization. So I can enable that tool for a cli sessions where I need some secrets but a rogue process that just tries to use the `op` binary isn't going to get to piggyback on that authorization. I'd get a new popup. But this is only step 1. Step 2 is...how to share that secret with the process that needs it and we are now back to the discussion above.
1: https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html#Environment= https://www.freedesktop.org/software/systemd/man/latest/syst...
- deleted 1y ago[deleted]
- robertlagrant 1y ago> On Linux systems, any user process can inspect any other process of that same user for it's environment variables. We can argue about threat model but, especially for a developer's system, there are A LOT of processes running as the same user as the developer. This is a very good point I'd never realised! I'm not sure how you get around it, though, as if that program can even find a credential and decrypt a file, if it runs as the user then everything else can go find that credential as well.
- rsyring 1y agoI added a note to my original post about how 1Password's cli tool work. Access to the binary doesn't mean automatic access to any authorized session. I think that's a good start. In my case, I have it tied into my fingerprint reader, so if something want's access to a secret using `op` I get a prompt to authorize. If I haven't just called `op` myself, I know it's a suspect request.
- chasil 1y agoIf a root exploit is leveraged, then /proc/*/environ of all processes is visible to the adversary. The classical alternative has been to store (FTP) credentials in a .netrc file (also used by curl). I have some custom code to pull passwords out of a SQLite database. For people who are really concerned with this, a "secrecy manger" is more appropriate, such as Cyberark conjur and summon, or Hashicorp Vault.
- jkrejcha 1y agoIf a full root exploit is leveraged, it's already game over already, basically anything at that point is just going to be rearranging deck chairs.
- robertlagrant 1y ago> If a root exploit is leveraged, then /proc/*/environ of all processes is visible to the adversary. > The classical alternative has been to store (FTP) credentials in a .netrc file (also used by curl). You're not suggesting that a netrc file would stop a root exploit?
- formerly_proven 1y ago--passphrase-fd was always the correct thing to do (gpg has had it for ages), but support for that is very meager. I mean, you can’t even kubectl login using environment variables, you pretty much have to pass tokens through command-line arguments and we’ve known that’s terrible for around forty-five years.
- bbkane 1y agoAny good cross-platform and easy ways to share secrets without using environment variables?
- zdc1 1y agoPoint to a file? E.g. `CONFIG_PATH=/etc/myapp/config.ini /opt/myapp` That being said, I still use env vars and don't plan on stopping. I just haven't (yet?) seen any exploits or threat models relating to it that would keep me up at night.
- maccard 1y agoHow do you get that file? I also use env vars.
- yencabulator 1y agoFor example, systemd or kubernetes. https://systemd.io/CREDENTIALS/ https://systemd.io/CREDENTIALS/ https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-files-from-a-pod https://kubernetes.io/docs/concepts/configuration/secret/#us... (Systemd also has more complex modes that are safer than files. And the Linux kernel has a concept of keyrings that might be even better.)
- deleted 1y ago[deleted]
- bbkane 1y agoIs that file more secure than the environment variables it's replacing? On Linux I think you can secure it to just your service with SELinux. Not sure about Windows
- matthew16550 1y agoSOPS can be part of the solution. It takes care of encrypting and decrypting config files. https://github.com/getsops/sops https://github.com/getsops/sops
- skylurk 1y agoI've used keyring, but ultimately does not solve the issue of every application running under the same user. https://pypi.org/project/keyring/ https://pypi.org/project/keyring/
- cedws 1y agoLinux security model is pretty broken without namespaces. systemd has some bells and whistles that help but if you want something better than environment variables you're naturally going to reach for cgroups.
- throwaway127482 1y agocgroups aren't relevant here, I think. Not sure if that was just a typo, since you did mention namespaces in the first sentence. PID and user namespaces in particular are relevant here.
- cedws 1y agoYeah seems you're right, I thought namespaces came under the cgroups umbrella.
- ux266478 1y agoLinux's namespaces are not a security mechanism. Please do not use them as one.
- cedws 1y agoI've said exactly the same thing on HN myself. However for this problem what do you suggest? Using namespaces is better than using no namespaces.
- ux266478 1y agoIf I was actually serious about the security of a system, I'd explicitly use virtualization at-minimum. If all I wanted was a mall cop and was happy to take the risk of misconfiguration or attacks on the kernel I'd probably say something that adds a lot of extra sanity to namespaces is "good enough", like bubblewrap. I would never trust it or say that it's secure though.
- jcgl 1y agoIs there consensus around that? Given how containers and bubblewrap/flatpak are considered (to some degree) security boundaries, I approach statements like this with skepticism.
- mhitza 1y ago> Any good solutions for passing secrets around that don't involve environment variables or regular plain text files? memfd_secret comes to mind https://man7.org/linux/man-pages/man2/memfd_secret.2.html https://man7.org/linux/man-pages/man2/memfd_secret.2.html I haven't seen much language support for it, though. On one part maybe because it's Linux only. People that write in Rust (and maybe Go, depends how easy FFI is) should give it a try. I wanted for a time to get some support for it in PHP, since wrapping a C function should be easy, but the thought of having to also modify php-fpm put a dent in that. I can't and don't want to hack on C code. In practice it'd be great if the process manager spawn children after opening a secret file descriptor, and pass those on. Not in visible memory, not in /proc/*/environ
- llimllib 1y agoFor go, it's available in /x/sys/unix: https://pkg.go.dev/golang.org/x/sys/unix#MemfdSecret https://pkg.go.dev/golang.org/x/sys/unix#MemfdSecret
- monocasa 1y agoOh, I like that. You should be able to build up a nice capability model to get access to those memfds from daemon too rather than having to spawn out of a process manager if that model fits your use case a bit better.
- deleted 1y ago[deleted]
- bjourne 1y agoIf are like most users you have secrets stored in read-protected files in ~/.ssh. There's nothing wrong with that.
- SoftTalker 1y agoThey are (should be) encrypted though, and not usable by someone who might somehow gain access to your ~/.ssh files.
- bjourne 1y agoNo, the private keys are not encrypted. If there were, where would you store the encryption key for decrypting them?
- SoftTalker 1y agoMy ssh private keys are encrypted. The keys are stored in my brain.
- amluto 1y agoSince at least 2012, environment variables have been at least as secure as ordinary memory: commit b409e578d9a4ec95913e06d8fea2a33f1754ea69 Author: Cong Wang <xiyou.wangcong@gmail.com> Date: Thu May 31 16:26:17 2012 -0700 proc: clean up /proc/<pid>/environ handling You can't read another process's environment unless you can ptrace-read the process, and if you can ptrace-read the process you know all its secrets anyway. cmdline is a different story.
- rsyring 1y agoI've never tried to use ptrace-read to read secrets from a running process's memory so I can't comment on that part. I had always assumed getting secrets from a running process' memory was non-trivial and/or required root permissions but maybe that's a bad assumption. However, reading a process' environment is as trivial as: `cat /proc/123456/environ` as long as it's ran by the same user. No ptrace-read required.
- intorio 1y agoYou do need PTRACE access to pid 123456 in order to access that file. It is transparent to you, but the kernel will use the current task's PTRACE_ATTACH access when attempting to get that information. By default, on most distributions, a user has PTRACE_ATTACH to all processes owned by it. This can be configured with ptrace_scope: https://www.kernel.org/doc/Documentation/security/Yama.txt https://www.kernel.org/doc/Documentation/security/Yama.txt
- marcosdumay 1y agoJust to clarify, that means that by default every process of the same user can access the variables. But it doesn't really matter because by default every process of the same user can read any secret directly from the target process anyway, right? And that the right thing to do if you want to harden your system is to disallow ptrace-read, and not bother changing software that uses environment variables? Because I think most people that just try will be able to read the variables of any process on their computer.
- vendiddy 1y agoNot an answer, but I do wish there was a low level primitive and a corresponding high level language construct to pass around secrets. Something like: my_secret = create_secret(value) Then ideally it's an opaque value from that point on
- 1718627440 1y agomy_secret = getenv ("VALUE"); :-)
- jkrejcha 1y agoUntil when? Secrets in applications in many cases (I would probably wager majority of the cases) are only useful if they're in plaintext at some point, for example if you're constructing a HTTP client or authenticating to some other remote system. As far as high-level language constructs go, there were similarish things like SecureString (in .NET) or GuardedString (in Java), although as best as I can tell they're relatively unused mostly because the ergonomics around them make them pretty annoying to use.
- vendiddy 1y agoJust seeing this now. The thinking was to minimize the the places where a secret could leak. So with an HTTP client, I would think at the lowest layer possible. I don't think of it as a way to eliminate secrets leaking. More-so reducing the surface area of accidental leaks.
- motorest 1y ago> Environment variables are often used to pass secrets around. But, despite its ubiquity, I believe that's a bad practice: I think environment variables are recommended to pass configuration parameters, and also secrets, in containerized applications managed by container orchestration systems. By design, other processes cannot inspect what environment variables are running in a container. Also, environment variables are passed to child processes because, by design, the goal is to run child processes in the same environment (i.e., same values) as the parent process, or with minor tweaks. Also, the process that spawns child processes is the one responsible for set it's environment variables, which means it already has at least read access to those secrets. All in all I think all your concerns are based on specious reasoning, but I'll gladly discuss them in finer detail.
- rsyring 1y agoPlease note that in my OP, I never mentioned containers. Let's say, as a developer, I need to do some API interactions with GitHub. So, in a terminal, using 1Password's cli tool `op`, I load my GH API token and pass it to my Python script that is doing the API calls. Presumably, the reason I use that process is because I want to keep that token's exposure isolated to that script and I don't want the token exposed on the filesystem in plaintext. There is no reason for every process running as my user on my laptop to have access to that token. But, that's not how things work. The Claude agent running in a different CLI session (as an example) also now has access to my GitHub token. Or, any extension I've ever loaded in VS Code also has access. Etc. It's better than having it in plain text on the file system but not by much. Additionally, unless I've misunderstood the systemd docs, if you are passing secrets through environment variables using unit's `Environment` config, ANY USER on a server can read those values. So now, we don't even have user isolation in effect. My reasoning is pretty plain. It could be wrong, but it's hardly specious.
- imiric 1y ago> The Claude agent running in a different CLI session (as an example) also now has access to my GitHub token. Or, any extension I've ever loaded in VS Code also has access. If you're giving untrusted software full access to your system, you're not in a position to complain about the system not being secure enough. Security starts by making mindful decisions according to your threat model. No system can keep you secure from yourself. I mean, it can, but it wouldn't be a system you have control over (see Apple, etc.). There are many solutions that can mitigate the risks you mention. They might not be as convenient as what you're doing now, but they exist. You can also choose to not use software that siphons your data, or exposes you to countless vulnerabilities, but that's a separate matter.
- jkrejcha 1y ago> - On Linux systems, any user process can inspect any other process of that same user for it's environment variables. We can argue about threat model but, especially for a developer's system, there are A LOT of processes running as the same user as the developer. Here's the thing though, the security model of most operating systems means that running a process as a user is acting as that user. There are some caveats (FreeBSD has capsicum, Linux has landlock, SELinux, AppArmor, Windows has integrity labels), but in the general case, if you can convince someone to run something, the program has delegated authority to act on behalf of that user. (And some user accounts on many systems may have authority to impersonate others.) While it's by no means the only security model (there exists fully capability based operating systems out there), it's the security model that is used for most forms of computing, for better or worse. One of the consequences of this is that you can control anything within your domain. I can kill my own processes, put them to sleep, and most importantly for this, debug them. Anything I own that has secrets can grab them my ptrace/process_vm_readv/ReadProcessMemory/etc.
- Narushia 1y ago> Any good solutions for passing secrets around that don't involve environment variables or regular plain text files? Honestly, my answer is still systemd-creds. It's easy to use and avoids the problem that plain environment variables have. It's a few years old by now, should be available on popular distros. Although credential support for user-level systemd services was added just a few weeks ago. A TL;DR example of systemd-creds for anyone reading this: # Run the initial setup systemd-creds setup # This dir should have permissions set to 700 (rwx------). credstore_dir=/etc/credstore.encrypted # For user-level services: # credstore_dir="$HOME/.config/credstore.encrypted" # Set the secret. secret=$(systemd-ask-password -n) # Encrypt the secret. # For user-level services, add `--user --uid uidhere`. # A TPM2 chip is used for encryption by default if available. echo "$secret" | systemd-creds encrypt \ --name mypw - "$credstore_dir/mypw.cred" chmod 600 "$credstore_dir/mypw.cred" You can now configure your unit file, e.g.: [Service] LoadCredentialEncrypted=mypw:/etc/credstore.encrypted/mypw.cred The process you start in the service will then be able to read the decrypted credential from the ephemeral file `$CREDENTIALS_DIR/mypw`. The environment variable is set automatically by systemd. You can also use the command `systemd-creds cat mypw` to get the value in a shell script. At least systemd v250 is required for this. v258 for user-level service support.
- mixmastamyk 1y agoYou've described the classic Unix security model, with minor improvements. While decent for the time it is showing its age. Specifically the difficulty in adapting to cheap, ubiquitous computing which it wasn't designed for. If you need to keep secrets from other processes—don't run them under the same user account. Or access them remotely, although that brings other tradeoffs and difficulties.
- lelanthran 1y ago> Any good solutions for passing secrets around that don't involve environment variables or regular plain text files? Plain text files are fine, it's the permissions on that file that is a problem. The best way is to be in control of the source to the program you want to run then you can make a change to always protect against leaking secrets by starting the program as a user that can read the secrets field. After startup the program reads the whole file and immediately drops privileges by switching to a user that cannot read the secrets file. Works for more than just secrets too.