5 ms·

> This is a list of some uses of environment variables from fairly widely used libraries and services. This shows that environment variables are pretty widely u

> This is a list of some uses of environment variables from fairly widely used libraries and services. This shows that environment variables are pretty widely used. Widely used, yes. Used as in read. Why do any of these need to change at runtime? And if they do - why are they environment variables? (NB: starting a new process is not "at runtime")

withinboredom 3y ago
Changing the env during runtime is actually quite handy for debugging and forcing the program into specific states. Other than that, it can also be handy in k8s with a VPA. You get more/less memory and then update the env to reflect that. Your service picks up the env change and updates the runtime. IIRC, there is/was some way to listen to those changes in C#, and automatically update runtime settings.
- eqvinox 3y ago
  > Other than that, it can also be handy in k8s with a VPA. You get more/less memory and then update the env to reflect that. Your service picks up the env change and updates the runtime. You… can't change the env from outside the process… are you saying this is used by disjoint components within a single process? Or is this just a misunderstanding?
  - withinboredom 3y ago
    You can spawn as many processes as you want in a container, did you not know that? But you only need access to the /proc/pid directory to change another processes env.
    - eqvinox 3y ago
      > But you only need access to the /proc/pid directory to change another processes env. /proc/$pid/environ is not writable (and as a matter of fact, due to how the environment works, it cannot be writable.)
      - LegionMammal978 3y ago
        But /proc/pid/mem is, if you like living dangerously! You'd just have to parse the dynamic-linker metadata to find where libc's environ is hiding. (Though statically-linked programs would be tougher.)
    - SAI_Peregrinus 3y ago
      Spawning a new process doesn't require changing the parent's environment.
  - pjc50 3y ago
    > You… can't change the env from outside the process… Not with that attitude you can't. (OK, without the joke: you can do this with an interactive debugger. But I think OP just meant "change it in the container and then restart the child process")
- rewmie 3y ago
  > Changing the env during runtime is actually quite handy for debugging and forcing the program into specific states. Most debuggers nowadays support altering variables at runtime after hitting breakpoints. In the meantime this was the very first time I ever heard anyone even considering changing env vars at runtime, let alone use it to debug stuff. Sounds like an ass-backwards way of going about debugging.
- riffraff 3y ago
  > Changing the env during runtime is actually quite handy for debugging and forcing the program into specific states. Wait, why would this need to happen at runtime? I have used env cars a lot to trigger specific cases but why would you want to do this while the process is running from within the process itself? If you control the process you can start it with the right env to begin with, no?
oefrha 3y ago
Have you ever exported anything in a shell script? Sure you can keep the necessary changes in local state and pass those to execve(2)/execvpe(3)/posix_spawn(3), and that would be safe AFAIK, but setenv(3) is there and more convenient if you're unaware of the hidden dangers. Also that doesn't work for PATH in execvp/execvpe, which is read from the current process; how do you change search paths for execvp without setenv (short of doing the search yourself)? Edit: I just realized macOS/FreeBSD has execvP() that allows passing a custom search path, so PATH is now safe, but without a -e variant, everything else is again unsafe.
- quickthrower2 3y ago
  Shell scripts are different as you are likely exporting environment variables and then starting new processes.
  - oefrha 3y ago
    Shell scripts aren’t different from “real” programs using exec or posix_spawn in this regard, it’s just that fewer people have done the latter than the former, so the former is a more relatable example. “Real” programs spawn other processes too you know, sometimes with modified environ.
    - quickthrower2 3y ago
      So I understand this right, I thought the issue is about multiple threads but in shell you wouldn’t have this just new processes. In a program you could have either.
- rcxdude 3y ago
  Shell scripts are not really prone to this problem because AFAIK no shells are multithreaded: subshells and the like are implemented with fork()
  - oefrha 3y ago
    Yes, I’m not saying shell scripts are affected, merely using them as an example to answer the question “Why do any of these [env vars] need to change at runtime?”
dmytroi 3y ago
Mostly integration, for example some library can only be configured via env variables, but a developer might want to configure it from with-in the app it's integrated into and used from. Also, few weeks ago I found a use for them when trying to pass configuration from Java/Kotlin to C++ library to be used during static constructors (invoked during dlopen) on Android, because at that phase native code cannot call back to JVM.
- guappa 3y ago
  > for example some library can only be configured via env variables library has already loaded when you call setenv, so what you're saying doesn't work in most cases. It seems to be a need to use poorly written libraries. You might consider fixing them instead.
  - jonhohle 3y ago
    I agree that would be a poor implementation, but the library could be loaded at runtime using dlopen or equivalent. This issue with that “interface” is the environment is process global. If the library is being loaded dynamically (specifically for some task) it would seem that the parameters are local to that task and should be taken by some reentrent init method. Alternatively, the process could be forked and environment set in the child without concern for thread safety or polluting the environment (think of the children!).
    - guappa 3y ago
      The only library I've seen to use env vars is libc, which uses them to decide how malloc should behave for example.
  - the_svd_doctor 3y ago
    Some libraries behaviour/API can be tweaked with env var. env var are read at runtime not loading time.
    - guappa 3y ago
wzdd 3y ago
Indeed -- it's an extremely unconvincing list, because any sensible library which may require a library user to set env variables (which includes all the ones I checked on the list) can also be configured without setting env variables. Most of the time the env variables set fallback defaults for parameters not specified by the caller. In these cases, the sane thing to do, regardless of the thread-safety of setenv(), is simply to supply the parameter in code. The only exception is things like debug logging, which is unlikely even to work dynamically. On the other hand, setenv() is clearly broken in modern code, particularly in a library context, and the man page (at least on my Linux machine) does not make that particularly obvious -- "Thread safety: MT-Unsafe" is the only note, with a reference to attributes(7) for more information. It could definitely be made more obvious.
qwertox 3y ago
Just asking: If you pass security tokens via environment variables to the process, doesn't it make sense to delete them from within the process after they have been used?
- eqvinox 3y ago
  Yes it would make sense, but no there is no way to actually ensure they have been deleted. A trivial but nonetheless very common case would be if your process is started with a wrapper shell script. But even just within your process, there is no guarantee at all against some random library (or the kernel) making a copy of the entire environment. If you want to pass secrets into a process at startup, I would strongly recommend passing a pipe as an additional open file descriptor (e.g. fd #4, but this FD number you can then put in an env variable) and writing it onto the pipe. It can only be read once, and you can control where the value propagates.
  - Zandikar 3y ago
    Damn, learning new tricks everyday, thanks for the tip.
  - yrro 3y ago
    Now if only someone would explain this to the authors of mongosh (which REFUSES to accept credentials in an environment variable and will ONLY read them from stdin or from argv...)
leoh 3y ago
Testing, for one thing… I mean YES you can factor your code (tests, whatever) to make this a non-issue but supposing some person wrote some code 10 years ago in an OSS project or on your team and you start banging into this issue. It’s not going to be trivial to unwind let alone find the root issue. Let’s start fixing things like this for our future selves, right? Digging heels in and saying “eh, you just got to learn this one weird quirk.. oh yeah this other one too..” is kind of a fun glass bead game until it’s not; as is not a winnning way to endear hearts and minds.

withinboredom 3y ago

Changing the env during runtime is actually quite handy for debugging and forcing the program into specific states. Other than that, it can also be handy in k8s with a VPA. You get more/less memory and then update the env to reflect that. Your service picks up the env change and updates the runtime. IIRC, there is/was some way to listen to those changes in C#, and automatically update runtime settings.

eqvinox 3y ago
> Other than that, it can also be handy in k8s with a VPA. You get more/less memory and then update the env to reflect that. Your service picks up the env change and updates the runtime. You… can't change the env from outside the process… are you saying this is used by disjoint components within a single process? Or is this just a misunderstanding?
- withinboredom 3y ago
  You can spawn as many processes as you want in a container, did you not know that? But you only need access to the /proc/pid directory to change another processes env.
  - eqvinox 3y ago
    > But you only need access to the /proc/pid directory to change another processes env. /proc/$pid/environ is not writable (and as a matter of fact, due to how the environment works, it cannot be writable.)
    - LegionMammal978 3y ago
      But /proc/pid/mem is, if you like living dangerously! You'd just have to parse the dynamic-linker metadata to find where libc's environ is hiding. (Though statically-linked programs would be tougher.)
  - SAI_Peregrinus 3y ago
    Spawning a new process doesn't require changing the parent's environment.
- pjc50 3y ago
  > You… can't change the env from outside the process… Not with that attitude you can't. (OK, without the joke: you can do this with an interactive debugger. But I think OP just meant "change it in the container and then restart the child process")
rewmie 3y ago
> Changing the env during runtime is actually quite handy for debugging and forcing the program into specific states. Most debuggers nowadays support altering variables at runtime after hitting breakpoints. In the meantime this was the very first time I ever heard anyone even considering changing env vars at runtime, let alone use it to debug stuff. Sounds like an ass-backwards way of going about debugging.
riffraff 3y ago
> Changing the env during runtime is actually quite handy for debugging and forcing the program into specific states. Wait, why would this need to happen at runtime? I have used env cars a lot to trigger specific cases but why would you want to do this while the process is running from within the process itself? If you control the process you can start it with the right env to begin with, no?

oefrha 3y ago

Have you ever exported anything in a shell script? Sure you can keep the necessary changes in local state and pass those to execve(2)/execvpe(3)/posix_spawn(3), and that would be safe AFAIK, but setenv(3) is there and more convenient if you're unaware of the hidden dangers. Also that doesn't work for PATH in execvp/execvpe, which is read from the current process; how do you change search paths for execvp without setenv (short of doing the search yourself)? Edit: I just realized macOS/FreeBSD has execvP() that allows passing a custom search path, so PATH is now safe, but without a -e variant, everything else is again unsafe.

quickthrower2 3y ago
Shell scripts are different as you are likely exporting environment variables and then starting new processes.
- oefrha 3y ago
  Shell scripts aren’t different from “real” programs using exec or posix_spawn in this regard, it’s just that fewer people have done the latter than the former, so the former is a more relatable example. “Real” programs spawn other processes too you know, sometimes with modified environ.
  - quickthrower2 3y ago
    So I understand this right, I thought the issue is about multiple threads but in shell you wouldn’t have this just new processes. In a program you could have either.
rcxdude 3y ago
Shell scripts are not really prone to this problem because AFAIK no shells are multithreaded: subshells and the like are implemented with fork()
- oefrha 3y ago
  Yes, I’m not saying shell scripts are affected, merely using them as an example to answer the question “Why do any of these [env vars] need to change at runtime?”

dmytroi 3y ago

Mostly integration, for example some library can only be configured via env variables, but a developer might want to configure it from with-in the app it's integrated into and used from. Also, few weeks ago I found a use for them when trying to pass configuration from Java/Kotlin to C++ library to be used during static constructors (invoked during dlopen) on Android, because at that phase native code cannot call back to JVM.

guappa 3y ago
> for example some library can only be configured via env variables library has already loaded when you call setenv, so what you're saying doesn't work in most cases. It seems to be a need to use poorly written libraries. You might consider fixing them instead.
- jonhohle 3y ago
  I agree that would be a poor implementation, but the library could be loaded at runtime using dlopen or equivalent. This issue with that “interface” is the environment is process global. If the library is being loaded dynamically (specifically for some task) it would seem that the parameters are local to that task and should be taken by some reentrent init method. Alternatively, the process could be forked and environment set in the child without concern for thread safety or polluting the environment (think of the children!).
  - guappa 3y ago
    The only library I've seen to use env vars is libc, which uses them to decide how malloc should behave for example.
- the_svd_doctor 3y ago
  Some libraries behaviour/API can be tweaked with env var. env var are read at runtime not loading time.
  - guappa 3y ago

wzdd 3y ago

Indeed -- it's an extremely unconvincing list, because any sensible library which may require a library user to set env variables (which includes all the ones I checked on the list) can also be configured without setting env variables. Most of the time the env variables set fallback defaults for parameters not specified by the caller. In these cases, the sane thing to do, regardless of the thread-safety of setenv(), is simply to supply the parameter in code. The only exception is things like debug logging, which is unlikely even to work dynamically. On the other hand, setenv() is clearly broken in modern code, particularly in a library context, and the man page (at least on my Linux machine) does not make that particularly obvious -- "Thread safety: MT-Unsafe" is the only note, with a reference to attributes(7) for more information. It could definitely be made more obvious.

qwertox 3y ago

Just asking: If you pass security tokens via environment variables to the process, doesn't it make sense to delete them from within the process after they have been used?

eqvinox 3y ago
Yes it would make sense, but no there is no way to actually ensure they have been deleted. A trivial but nonetheless very common case would be if your process is started with a wrapper shell script. But even just within your process, there is no guarantee at all against some random library (or the kernel) making a copy of the entire environment. If you want to pass secrets into a process at startup, I would strongly recommend passing a pipe as an additional open file descriptor (e.g. fd #4, but this FD number you can then put in an env variable) and writing it onto the pipe. It can only be read once, and you can control where the value propagates.
- Zandikar 3y ago
  Damn, learning new tricks everyday, thanks for the tip.
- yrro 3y ago
  Now if only someone would explain this to the authors of mongosh (which REFUSES to accept credentials in an environment variable and will ONLY read them from stdin or from argv...)

leoh 3y ago

Testing, for one thing… I mean YES you can factor your code (tests, whatever) to make this a non-issue but supposing some person wrote some code 10 years ago in an OSS project or on your team and you start banging into this issue. It’s not going to be trivial to unwind let alone find the root issue. Let’s start fixing things like this for our future selves, right? Digging heels in and saying “eh, you just got to learn this one weird quirk.. oh yeah this other one too..” is kind of a fun glass bead game until it’s not; as is not a winnning way to endear hearts and minds.