6 ms·

Yet another person is burned by calling setenv() in a multi-threaded context. There really needs to be a big warning banner on the manpage for setenv() that wa

Yet another person is burned by calling setenv() in a multi-threaded context. There really needs to be a big warning banner on the manpage for setenv() that warns about this because it seems like a far more common problem than you would expect.

01HNNWZ0MV43FF 2y ago
Funny enough, the Rust wrapper `std::env::set_var` does have a big warning https://doc.rust-lang.org/std/env/fn.set_var.html https://doc.rust-lang.org/std/env/fn.set_var.html
- subarctic 2y ago
  Looks like that Safety section was added in 1.76.0. It'll be an even bigger warning in the future since it's now going to be unsafe in Rust 2024
umpalumpaaa 2y ago
The man page says: > POSIX.1 does not require setenv() or unsetenv() to be reentrant. A non-reentrant function cannot be thread safe. In general (for POSIX, libc and many other libraries: if the docs do not explicitly say "this function is thread safe" they are not).
- wmf 2y ago
  It's time to move beyond this attitude and make things safe by default. For example, Solaris has a safer version of setenv(). "It is ridiculous that this has been a known problem for so long. It has wasted thousands of hours of people's time, either debugging the problems, or debating what to do about it. We know how to fix the problem." https://www.evanjones.ca/setenv-is-not-thread-safe.html https://www.evanjones.ca/setenv-is-not-thread-safe.html
  - umpalumpaaa 2y ago
    I am not sure making things safe by default is a good idea. This always comes with a cost. Thats also the reason why basic data types (array, dictionaries, etc) are generally not thread safe… because its usually not needed or handled on a much higher level. Its a different story for languages/environments that are supposed to be safe by default and where you have language features that ensure safety (actors, optionals etc) but not for something like libc which has a standard it has to conform to and like 100 years of history.
    - dgrunwald 2y ago
      The problem with `setenv` is that people expect one process to have one set of environment variables, which is shared across multiple languages running in that process. This implies every language must let its environment variables be managed by a central language-independent library -- and on POSIX systems, that's libc. So if libc refuses to provide thread-safety, that impacts not just C, but all possible languages (except for those that cannot call into C-libraries; as those don't need to bother synchronizing the environment with libc).
      - PaulDavisThe1st 2y ago
        It's not just that "libc refuses to provide thread-safety" ... the POSIX standard specifies that these functions are non-reentrant.
        saagarjha 2y ago
        A conformant implementation can make a non-reentrant function actually safe under the hood for people that call into it erroneously. Unfortunately, there is no way to do this for getenv/setenv, because of the API they expose (specifically, when environ is accessed directly).
    - tsimionescu 2y ago
      In some cases this is true. In the case of setting and getting env vars, it is not. There is no comceivable reason for making a process that spends any significant portion of its runtime calling setenv() or getenv(). Even if those calls were a thousand times slower than today, it would still be a non-issue.
- deleted 2y ago
  [deleted]
- jabl 2y ago
  > A non-reentrant function cannot be thread safe. Actually, a non-reentrant function can be thread-safe. A common example of such a function in libc being malloc().
  - adrian_b 2y ago
    By definition, a "reentrant function" is a function that may be invoked even when it has not returned yet from a previous invocation. So a non-reentrant function is a function that may not be invoked again between a previous invocation and returning from that invocation. When a function may be invoked from different threads, then it is certain that sometimes it will be invoked by a thread before returning from a previous invocation from a different thread. Therefore any function that may be invoked from different threads must be reentrant. Otherwise the behavior of the program is unpredictable. Reentrant functions may be required even in single-thread programs, when they may be invoked recursively, or they may be invoked by signal handlers. An implementation of "malloc" may be reentrant or it may be non-reentrant. Old "malloc" implementations were usually non-reentrant because they used global variables for managing the heap. Such "malloc" functions could not be used in multi-threaded programs. Modern "malloc" implementations are reentrant, either by using only thread-local storage or by using shared global variables to which some method for concurrent access is implemented, e.g. with mutual exclusion.
    - tedunangst 2y ago
      Who has a signal safe malloc?
      - adrian_b 2y ago
        POSIX does not require malloc to be signal safe. Therefore I do not think that anyone has bothered to implement a signal-safe malloc, as this is likely to be complicated. Allocating memory in a signal handler makes no sense in a well designed program, so not being allowed to use malloc and related functions is not a problem.
        2y ago

01HNNWZ0MV43FF 2y ago

Funny enough, the Rust wrapper `std::env::set_var` does have a big warning https://doc.rust-lang.org/std/env/fn.set_var.html https://doc.rust-lang.org/std/env/fn.set_var.html

subarctic 2y ago
Looks like that Safety section was added in 1.76.0. It'll be an even bigger warning in the future since it's now going to be unsafe in Rust 2024

umpalumpaaa 2y ago

The man page says: > POSIX.1 does not require setenv() or unsetenv() to be reentrant. A non-reentrant function cannot be thread safe. In general (for POSIX, libc and many other libraries: if the docs do not explicitly say "this function is thread safe" they are not).

wmf 2y ago
It's time to move beyond this attitude and make things safe by default. For example, Solaris has a safer version of setenv(). "It is ridiculous that this has been a known problem for so long. It has wasted thousands of hours of people's time, either debugging the problems, or debating what to do about it. We know how to fix the problem." https://www.evanjones.ca/setenv-is-not-thread-safe.html https://www.evanjones.ca/setenv-is-not-thread-safe.html
- umpalumpaaa 2y ago
  I am not sure making things safe by default is a good idea. This always comes with a cost. Thats also the reason why basic data types (array, dictionaries, etc) are generally not thread safe… because its usually not needed or handled on a much higher level. Its a different story for languages/environments that are supposed to be safe by default and where you have language features that ensure safety (actors, optionals etc) but not for something like libc which has a standard it has to conform to and like 100 years of history.
  - dgrunwald 2y ago
    The problem with `setenv` is that people expect one process to have one set of environment variables, which is shared across multiple languages running in that process. This implies every language must let its environment variables be managed by a central language-independent library -- and on POSIX systems, that's libc. So if libc refuses to provide thread-safety, that impacts not just C, but all possible languages (except for those that cannot call into C-libraries; as those don't need to bother synchronizing the environment with libc).
    - PaulDavisThe1st 2y ago
      It's not just that "libc refuses to provide thread-safety" ... the POSIX standard specifies that these functions are non-reentrant.
      - saagarjha 2y ago
        A conformant implementation can make a non-reentrant function actually safe under the hood for people that call into it erroneously. Unfortunately, there is no way to do this for getenv/setenv, because of the API they expose (specifically, when environ is accessed directly).
  - tsimionescu 2y ago
    In some cases this is true. In the case of setting and getting env vars, it is not. There is no comceivable reason for making a process that spends any significant portion of its runtime calling setenv() or getenv(). Even if those calls were a thousand times slower than today, it would still be a non-issue.
deleted 2y ago
[deleted]
jabl 2y ago
> A non-reentrant function cannot be thread safe. Actually, a non-reentrant function can be thread-safe. A common example of such a function in libc being malloc().
- adrian_b 2y ago
  By definition, a "reentrant function" is a function that may be invoked even when it has not returned yet from a previous invocation. So a non-reentrant function is a function that may not be invoked again between a previous invocation and returning from that invocation. When a function may be invoked from different threads, then it is certain that sometimes it will be invoked by a thread before returning from a previous invocation from a different thread. Therefore any function that may be invoked from different threads must be reentrant. Otherwise the behavior of the program is unpredictable. Reentrant functions may be required even in single-thread programs, when they may be invoked recursively, or they may be invoked by signal handlers. An implementation of "malloc" may be reentrant or it may be non-reentrant. Old "malloc" implementations were usually non-reentrant because they used global variables for managing the heap. Such "malloc" functions could not be used in multi-threaded programs. Modern "malloc" implementations are reentrant, either by using only thread-local storage or by using shared global variables to which some method for concurrent access is implemented, e.g. with mutual exclusion.
  - tedunangst 2y ago
    Who has a signal safe malloc?
    - adrian_b 2y ago
      POSIX does not require malloc to be signal safe. Therefore I do not think that anyone has bothered to implement a signal-safe malloc, as this is likely to be complicated. Allocating memory in a signal handler makes no sense in a well designed program, so not being allowed to use malloc and related functions is not a problem.
      - 2y ago