5 ms·
Twelve factor app: use environment for config Docker: Don't use environment for config, it's insecure I like and use many of the patterns of 12 factor, but in
by sergioisidoro 3y ago
Twelve factor app: use environment for config
Docker: Don't use environment for config, it's insecure
I like and use many of the patterns of 12 factor, but indeed some of these were written in the context of VPS, where environment is stable, secure and more constant (unlike containers, where environment might end up shipped in one of the layers). In the age of containers, this particular point has been a bit of a struggle for me. Docker secrets don't always play well, and you need to do some gymnastics to make it work.
- deleted 3y ago[deleted]
- samus 3y agoThe basic idea is still valid. You might just have to use a more appropriate mechanism when using containers. For example, the explanation of the Config factor explains that a config file can also be used.
- 89vision 3y agodo you have any examples of more secure ways of injecting secrets?
- dharmab 3y agoCheck out the Secrets Operator for Kubernetes. Injects your secrets from a secrets store as a file mounted into your container.
- marwis 3y agoAnd how is /var/run/secrets any more secure than env? I guess you avoid the risk of accidentally logging secrets with other env variables but otherwise it seems to be just as secure/insecure.
- deleted 3y ago[deleted]
- dharmab 3y agoThe secrets aren't in a shared location. They're stored in an ephemeral volume specific to each container which other processes cannot access.
- marwis 3y agoIf someone has enough privileges to access /proc/*/environ of another process (i.e. root or the same user or process or child process) then they should be easily able to reach inside the container, no?
- lmm 3y agoOther processes running as the same user can be blocked from reading /var/run/secrets via the likes of SELinux/AppArmor whereas they can't be blocked from reading /proc/environ. It's a pretty fine distinction and I don't know how many people actually bother doing SELinux etc. in practice, but theoretically it's marginally better.
- marwis 3y agoAppArmor can restrict /proc, see example from docker: https://github.com/moby/moby/blob/master/contrib/apparmor/template.go#L41 https://github.com/moby/moby/blob/master/contrib/apparmor/te...
- 89vision 3y agohow does the application pick it up? We use the built in secrets that are injected into the container as env vars and then the application picks it up that way. Not trying to sound combative, just looking for better ways to do things.
- jameshart 3y agoRead the file on startup. Bonus: you can watch the file for changes. Which means your app can pick up rotated secrets without a process restart, whereas if you inject secrets via the environment they're fixed for process lifetime.
- 3by7 3y agoIn retrospect, a better name for that section could have been "Separate config from code" as it says in one of the first paragraphs.
- chrissoundz 3y agoHow are environment variables insecure?
- dharmab 3y agoEnvvars have much weaker access controls than files; basically anything in the same PID namespace can read your envvars. Poke around in /proc/*/environ to see for yourself. Files can use user permissions and MAC (AppArmor, SELinux, etc) to secure them against unauthorized processes.
- wutwutwat 3y agoIf running in a containerized environment ENV is injected only to the container, at container runtime. If you’re running your containers with a single process, you are doing the exact access control you just outlined. At that point what is the difference between a file on disk with read permissions for the proc owner vs only the proc having the ENV injected to its shell at runtime? I’d maybe even say the ENV at runtime is more secure than the chown’d file on disk because the ENV injected variant is ephemeral and when that container/proc dies, so does the ENV sitting in its memory, whereas the other leaves a file on disk potentially depending on where it was written to, regardless of who can access it. Then there’s the fact that a lot more people than you think run containers as root so all file access control goes out the window. From a security perspective it is always more secure to have something only in memory of the running process than it is to have a file on disk regardless of file permissions
- dharmab 3y agoContainers don't enforce that isolation. Another process can nsenter the container's PID namespace. You also have issues if your container's PID1 creates subprocesses. File are not necessarily written to disk. e.g. the Secrets CSI Driver loads secrets directly from a secrets store as virtual files within ephemeral volumes.
- ninkendo 3y ago
- jcrites 3y agoCould you elaborate? How are environment variables insecure? If the secret will be present in the application throughout it's lifetime, then what's a better alternative? Just because a secret is injected into the application as an environment variable does not mean it's handled insecurely elsewhere. For example, AWS ECS containers have built-in support for fetching secrets from Secret Manager at startup and passing them as environment variables: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/secrets-envvar.html https://docs.aws.amazon.com/AmazonECS/latest/developerguide/... This fetches the secret at the time the container starts up (IIRC), and uses the IAM credentials of the running application to fetch them from Secret Manager (so it must have permission to the secret). The merit of using environment variables seems to depend entirely on how they are, well, configured. With a mechanism like this I don't see much disadvantage. (The main disadvantage I see is that generic malware that attempts to dump environment variables might capture them, but defending against that threat is largely obfuscation, unless you aren't storing the secret at all permanently in memory.)
- dathinab 3y ago- any sub process has full unlimited access to env variables - env variables are prone to be dumped in various contexts and might leak secrets to e.g. logs - env variables might leak, e.g. in context of docker build stages etc. (but then using building a docker image to build both the image and the software is often anyway not a good idea) - env variables are often easily accessible by other processes the method you mentioned of ad-hoc injecting a secret as env by some form of security manager for containers avoid many such problems
- lazyant 3y ago> - env variables are often easily accessible by other processes A process would need to have the same permission (same user) or root to read /proc/$PID/environ no? or what mechanism is there for a process to read another process env vars (not children processes, that was already mentioned)? unless of course, a process dumped them in some world-readable file (like logs as mentioned) or some other silly way
- dathinab 3y agoit's both true putting any form of secrets into a env var is deeply insecure using env is the most simple uniformly available way to have configs through I would take it a step further, anything which must not be leaked probably should not be in any config, weather env or file or similar, it should be injected through other means if viable
- 0cf8612b2e1e 3y agoOn the other hand, without platform support there are not a lot of easy alternatives. It at least encourages separation of config and code, keeping secrets out of vcs.
- andreimackenzie 3y agoAgree. The main benefit of environment variables is portability. They work pretty much the same way in every platform I have encountered. As soon as a system gets into files, it's possible to get bogged down in the complexity of Windows vs. Unix, file permissions, SELinux, etc. The simplicity is often worthwhile over the extra security theoretically possible with well-managed files.
- wutwutwat 3y agoOne could argue that you should never consider any input as “safe to consume”, and if you write defensive app code, would never consider the ENV to ever be secure or what you expect, and would therefor sanitize that input before consuming. If you’re blindly trusting any input just because you’re single tenant, you’re in for some hard to track down bugs and long nights when those circumstances one day change due to random business pivots.
- _ea1k 3y agoYeah, I've never been a fan of secrets in env vars for this reason. It is shocking how frequent dumb debug consoles have accidentally exposed env vars. You really don't want that to accidentally expose your private keys. There's really nothing wrong with secrets mounted on the file system via k8s. But, of course, all of this depends on your deployment environment. Sometimes env vars are the least bad option, because secrets really are always hard.
- berkes 3y ago> dumb debug consoles have accidentally exposed env vars Huh? If the app can read the secret, then the app can... read the secret. Maybe I'm misunderstanding what you call a "debug console", but if this is anything running in the app, it can ... read the secrets. Regardless of their source. Or, to put it differently: it matters nothing if your secrets at rest are stored in a quantum-resistent-encrypted vault, secrets.yml or ENV vars, if your logging, backtracing or debug consoles dump them somewhere, you are compromised regardless.
- _ea1k 3y agoThis is absolutely true, but it is far more common for someone to accidentally leave open a display of env vars than a display of files. I've got this problem to debug in prod, I know, I'll embed the env vars in the source code of the error page. No big deal and it might help me debug that! For a dev used to not using secrets in env vars (for example, when testing locally), this is an easy mistake. He may even use a third party library that embeds features like this without knowing it! Even things like the jmx console can be risky, since access to that may not relate 1:1 with access to secrets. I've never seen anyone make a similar mistake and accidentally embed all_my_passwords.ini in their output. I must admit that some of what I've said above has become outdated, since it is now a much more common practice to embed passwords in env vars than it used to be.
- berkes 3y agoWholeheartedly disagree. I've yet to come across any git repo that didn't, at some point, commit the all_mypasswords.ini (or, just as bad, the .env.backup). I've only once ever seen a repo that had ENV vatlrs committed (where some moron committed the entire ./dist directory which included a Json that held the copy of ENV at compile time) My point is not that "one is obviously better", but that none is obviously better. All solutions have downsides. Implementing 12 factor means understanding those.