6 ms·

Why I don't like environment variables: 1. I worry about programs dumping all their environment variables to log files - credentials are now on disk, ingested

• bopbeepboop 6y ago
  How do command line arguments or config files solve either problem?
  • yrro 6y ago
    Command line arguments aren't inherited by child processes. Unfortunately they are visible to other users on the system, so they're no good for credentials. Config files (or an abstraction of them such as reading config data from a socket), after parsing, result in some credentials sitting in the memory of the process that needs them. They aren't in the environment block, so a quick and dirty "dump all my environment variables to stdout" procedure won't risk exposing them. And for the same reason, a child process that does the same won't inherit them in order to expose them. Note that talking about preventing accidental exposure of credentials. Config files alone can't protect credentials from a malicious process that deliberately goes looking for them to leak them; for that additional measures have to be taken... but environment variables aren't part of the solution!
    • bopbeepboop 6y ago
      > Unfortunately they are visible to other users on the system, so they're no good for credentials. If you’re concerned about your own software logging credentials, command line arguments are negative in two regards: They’re highly visible when the process is running; they’re often automatically logged. > They aren't in the environment block, so a quick and dirty "dump all my environment variables to stdout" procedure won't risk exposing them. Okay — but the usual way that happens is “dump my config object in a log”, which parsed configs don’t help with. You also now have a config file: how is it stored? ...is it in the repo? ...what are the permissions? ...how do we deploy it? Environment variables don’t persist in repos and are designed to be integrated with hosting tools, like secrets managers. I’m not seeing how a config file beats Kubernetes injecting from the secret store, which is why we use environment variables: so our tools (secret stores) can configure the environment our software uses.
      • yrro 6y ago
        Good point about command line arguments being often automatically logged! So they bad for both reasons :) Now, if you're running in k8s then you can improve your setup by mounting your secret into your container, and have your code read the credentials from the file within the mount. This just looks like another kind of config file to me :)
        • PeterisP 6y ago
          Embedding secrets (which should be changeable and with limited access) into container images (which should be reproducible and perhaps stored in accessible locations) sounds like not a goood idea; IMHO you definitely need the capability to have the same container use different credentials so that, for example, you can run the same container in a development or testing environment as in production, but with different credentials.
          • bopbeepboop 6y ago
            I believe they meant mounting via Kubectrl as a file, from the secret manager. So runtime file injection.
          • yrro 6y ago
            I'm talking about doing this: https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-files-from-a-pod https://kubernetes.io/docs/concepts/configuration/secret/#us...
        • bopbeepboop 6y ago
          > And for the same reason, a child process that does the same won't inherit them in order to expose them. Wait, this is the crux of why you think it’s more secure — but actually I see the reverse problem: Dropping environmental variables is standard security practice, but dropping file access permissions is not. Most child processes read from the same set of files as the parents. How would having files rather than ENVs make my container more secure, where we’re concerned with developers making mistakes (passing ENV vs passing file permissions)? Similarly, the only proposed benefit of your idea is we don’t have them around post reading — but that’s true if you initialize a config object from ENV and then pass it around as well. (You ignored my point about how mistakes via logging happen.)
    • hajile 6y ago
      It's also easier to encrypt a config file and provide the decrypt key externally.
      • PeterisP 6y ago
        So then the real secret is the key, which is provided "externally" ... how? Through command line parameters, some other config file, or environment variables? :P
        • IMTDb 6y ago
          Another encrypted config file. Obviously.
          • snicker7 6y ago
            Encrypted files all the way down.
        • Someone 6y ago
          A file that’s protected appropriately or standard input.
• intrepidhero 6y ago
  Isn't storing credentials in environmental variables bad practice to begin with?
  • jrwr 6y ago
    depends on who you are talking to, since you run the risk of committing the creds to a git repo
  • darksaints 6y ago
    Compared to what?
  • yrro 6y ago
    Yes, for those reasons
  • 0xEFF 6y ago
    What better place is there to store credentials?
    • whatshisface 6y ago
      Store a path to the top secret file in an environment variable, and have the program read the credentials out of the file. Put the file somewhere far away from the repo, on the deployed filesystem.
    • falcolas 6y ago
      A config store like Vault. Of course, that needs credentials too, which are typically a file on the file system. IMO, people are overly sensitive about environment vars. They are really no worse than files on the file system - both can be accessed if you're a privileged user on that machine.
      • KptMarchewa 6y ago
        Vault should be source of those env variables. Via some predefined initcontainer or something like that, to which devs don't have access to.
• npsimons 6y ago
  My first thought on the headline are specialized concerns of the above: environment variables are an attack surface. If you use them for configuration, it's all too easy for an attacker to modify them without the victim knowing. Just look at issues with LD_PRELOAD: https://attack.mitre.org/techniques/T1574/006/ https://attack.mitre.org/techniques/T1574/006/ That said, I agree with GP that environment variables are super useful and super simple. But I've also been burned more than a couple of times by setting something in the past and then having it caused unexpected bugs that are hard to trace down as they aren't in my working memory. They're a double-edged sword, to be sure.
  • jschwartzi 6y ago
    There's actually a long list of variables that are unset when invoking sudo to prevent these kinds of attacks. Systemd will also start programs with a very minimal environment that isn't inherited from any shells. You then have to specify environment variables explicitly as part of the unit file. You can also specify environment variables in environment files.
• OJFord 6y ago
  I used to agree with (2), but now I think Meh, it's an implementation detail whether the program uses my environment variable 'directly' or with a child process, it's not meaningful to make that distinction. When it is meaningful (and this is supported today) is to set them just for specific programs/invocations, rather than exporting for a long-running interactive shell (and everything within it) willy-nilly. More innovation around making that easier would be interesting, env vars that should be set specified by program, isolated from others, for example. So `foobar` would actually get executed like `FOO_SECRET=hunter2 foobar` without specifying it every time or having it exported in the shell, and in a generic way not specific to each program's config. It's not really related but for some reason 'summon' is on my mind as a tool to mention. I haven't used it in anger yet, but it is interesting. It's not quite this though, or at least, it solves only the 'storage' part of the implementation of what I described, not the 'orchestration' or mapping of programs to vars/summon invocations.
  • Lvl999Noob 6y ago
    Iiuc you are advocating for setting env vars at the call site, like `FOO_SECRET=hunter2 foobar`? In that case, why not just use command-line args and call it like `foobar --secret=hunter2`?
    • OJFord 6y ago
      I wasn't advocating for it in preference to args, but there are circumstances where that's not possible, for example calling some CI/CD tool (say terraform, ansible, fabric, whatever) that doesn't consume the var itself but uses something that does. It's also a more convenient/already generic interface for doing something consistent across multiple programs.
    • PeterisP 6y ago
      Because in the latter case the commandline of the executed process (which may be exposed in various places, including a simple process list) is `foobar --secret=hunter2` and in the former case it's just `foobar`.
  • jschwartzi 6y ago
• rascul 6y ago
  > I wish the ghosts of unix past had forseen the need for a way to mark particular variables as, say, 'sensitive' and 'noexport', allowing them to opt out of the default behaviour. The default behavior is a non-exported variable. If you want child processes to see it, you must export it.
  • yrro 6y ago
    There is no such thing as an exported or non-exported environment variable. In fact, as the kernel is concerned, there is no such thing as an environment _variable_ at all, just a block of data. See execve(2): "envp is an array of pointers to strings, conventionally of the form key=value, which are passed as the environment of the new program. The envp array must be terminated by a NULL pointer." You can confirm this my examining the the environment block that was passed in to your current shell with: < /proc/$$/environ tr '\0' '\n' What you're referring to are actually "shell parameters", some of which may be marked for export. When the shell starts up, it parses the environment block and sets parameters based on what it finds, marking them all for export. And the shell uses only the parameters marked for export when constructing the environment block for a child process (which is passed to execve(2)/execveat(2) in the envp argument).
    • rascul 6y ago
      I guess I assumed you were referring to environment variables in the context of the shell. Apparently I was incorrect.
• HelloNurse 6y ago
  1. If your program is chatty, it can be chatty in the same way regardless of where the improperly logged secrets come from; it's still your fault for being coarse and lazy. There's little difference between logging all environment variables(and/or all command line parameters) and logging the whole configuration object. 2. If your child processes shouldn't inherit environment variables, set them properly. The "ghosts of Unix past" have "foreseen the need" for execve(2) and execveat(2), which don't pass anything by "default".