5 ms·

The man page says: > POSIX.1 does not require setenv() or unsetenv() to be reentrant. A non-reentrant function cannot be thread safe. In general (for

The man page says: > POSIX.1 does not require setenv() or unsetenv() to be reentrant. A non-reentrant function cannot be thread safe. In general (for POSIX, libc and many other libraries: if the docs do not explicitly say "this function is thread safe" they are not).

wmf 2y ago
It's time to move beyond this attitude and make things safe by default. For example, Solaris has a safer version of setenv(). "It is ridiculous that this has been a known problem for so long. It has wasted thousands of hours of people's time, either debugging the problems, or debating what to do about it. We know how to fix the problem." https://www.evanjones.ca/setenv-is-not-thread-safe.html https://www.evanjones.ca/setenv-is-not-thread-safe.html
- umpalumpaaa 2y ago
  I am not sure making things safe by default is a good idea. This always comes with a cost. Thats also the reason why basic data types (array, dictionaries, etc) are generally not thread safe… because its usually not needed or handled on a much higher level. Its a different story for languages/environments that are supposed to be safe by default and where you have language features that ensure safety (actors, optionals etc) but not for something like libc which has a standard it has to conform to and like 100 years of history.
  - dgrunwald 2y ago
    The problem with `setenv` is that people expect one process to have one set of environment variables, which is shared across multiple languages running in that process. This implies every language must let its environment variables be managed by a central language-independent library -- and on POSIX systems, that's libc. So if libc refuses to provide thread-safety, that impacts not just C, but all possible languages (except for those that cannot call into C-libraries; as those don't need to bother synchronizing the environment with libc).
    - PaulDavisThe1st 2y ago
      It's not just that "libc refuses to provide thread-safety" ... the POSIX standard specifies that these functions are non-reentrant.
      - saagarjha 2y ago
        A conformant implementation can make a non-reentrant function actually safe under the hood for people that call into it erroneously. Unfortunately, there is no way to do this for getenv/setenv, because of the API they expose (specifically, when environ is accessed directly).
  - tsimionescu 2y ago
    In some cases this is true. In the case of setting and getting env vars, it is not. There is no comceivable reason for making a process that spends any significant portion of its runtime calling setenv() or getenv(). Even if those calls were a thousand times slower than today, it would still be a non-issue.
- cogman10 2y ago
  You can't. You could wrap setenv in a mutex, but that's not good enough. It can still be called from different processes, which means you'd need to do a more expensive and complex syncing system to make it safe. That ballons out to other env related methods needing to honor the synchronization primitive in order for there to be a semblance of safety. However, you still end up in a scenario where you can call setenv getenv and that would be incorrect because between the set and the get, even with mutexes properly in place and coordinated amongst different applications, you have a race condition where your set can be overwritten by another application's set before your get can run. Now, instead of actually making these functions safe you've buried the fact that external processes (or your own threads) can mess with env state. The solution is to stop using env as some sort of global variable and instead treat it as a constant when the application starts. Using setenv should be mostly discouraged because of these issues.
  - ryao 2y ago
    How does an external process mess with env state? As far as I know, you pass the environment when doing the execvpe() and then you cannot touch it from outside of the process anymore.
    - jenadine 2y ago
      You're correct. Parent comment is inaccurate. The problem is that a different library in the same process can use getenv without locking (or without locking the same lock as your code)
  - tsimionescu 2y ago
    Of course you can. Mutexes are system objects, so it's not a huge problem to sync across processes, if you really have to (is it really expected that one process can set env vars inside another process?). Making global state, especially state that has no reason to be modified or even read very often like the env, thread safe is a trivial issue, well studied and understood. Could an intern do it? Probably not. Could literally any maintainer of a standard C library? Easily. This is much more of a culture problem preventing such obvious flaws from being recognized as such. Side-note: your set-then-get example is a theoretical problem in search of a use case. Why would you ever want to concurrently set an env var and expect to be guaranteed to read that same value? And even if this is a real thing that applications really use, exposing a new function to sync anything on the env mutex is, again, trivial. So, if you really needed that, you could do lockenv setenv getenv unlockenv And problem solved.
- PaulDavisThe1st 2y ago
  One of the major differences between X Window and the win32 GUI APIs is that the windows one builds in thread safety, and it cannot be removed. This means that you pay the price of mutexes and the like (what the windows world likes to call "critical sections"), even if you have a single threaded GUI. X Window, on the other hand, decided to do nothing about threads at all, leaving it up to the application. 30 years after these decisions were made, most sensible people do single threaded GUIs anyway (that is, all calls to the windowing API come from a single thread, and all redraws occur synchronously with respect to that thread; this does not block the use of threads functioning as workers on behalf of the GUI, but they are not allowed to make windowing API calls themselves). Consequently, the overhead present in the win32 API is basically just dead-weight, there to make sure that "things are safe by default". There's a design lesson here for everyone, though precisely what it is will likely still be argued about.
  - wmf 2y ago
    Yet 30 years later people are calling setenv()/getenv() from different threads even though "it is known" that it crashes. For whatever reason the lesson from GUIs doesn't apply here.
    - PaulDavisThe1st 2y ago
      Judging from a lot of the comments in this thread, the idea that there could even be parts of the *POSIX API* that are not thread-safe seems like an idea that hasn't even occured to a lot of (younger?) programmers ...
  - lmm 2y ago
    > Consequently, the overhead present in the win32 API is basically just dead-weight, there to make sure that "things are safe by default". How much overhead is it though? IIRC uncontended mutexes are practically free, especially when they're only being used from a single thread. Our industry is way too eager to make things unsafe for the sake of marginal performance differences that are irrelevant for most use cases, IMO.
deleted 2y ago
[deleted]
jabl 2y ago
> A non-reentrant function cannot be thread safe. Actually, a non-reentrant function can be thread-safe. A common example of such a function in libc being malloc().
- adrian_b 2y ago
  By definition, a "reentrant function" is a function that may be invoked even when it has not returned yet from a previous invocation. So a non-reentrant function is a function that may not be invoked again between a previous invocation and returning from that invocation. When a function may be invoked from different threads, then it is certain that sometimes it will be invoked by a thread before returning from a previous invocation from a different thread. Therefore any function that may be invoked from different threads must be reentrant. Otherwise the behavior of the program is unpredictable. Reentrant functions may be required even in single-thread programs, when they may be invoked recursively, or they may be invoked by signal handlers. An implementation of "malloc" may be reentrant or it may be non-reentrant. Old "malloc" implementations were usually non-reentrant because they used global variables for managing the heap. Such "malloc" functions could not be used in multi-threaded programs. Modern "malloc" implementations are reentrant, either by using only thread-local storage or by using shared global variables to which some method for concurrent access is implemented, e.g. with mutual exclusion.
  - tedunangst 2y ago
    Who has a signal safe malloc?
    - adrian_b 2y ago
      POSIX does not require malloc to be signal safe. Therefore I do not think that anyone has bothered to implement a signal-safe malloc, as this is likely to be complicated. Allocating memory in a signal handler makes no sense in a well designed program, so not being allowed to use malloc and related functions is not a problem.
      - 2y ago

wmf 2y ago

It's time to move beyond this attitude and make things safe by default. For example, Solaris has a safer version of setenv(). "It is ridiculous that this has been a known problem for so long. It has wasted thousands of hours of people's time, either debugging the problems, or debating what to do about it. We know how to fix the problem." https://www.evanjones.ca/setenv-is-not-thread-safe.html https://www.evanjones.ca/setenv-is-not-thread-safe.html

umpalumpaaa 2y ago
I am not sure making things safe by default is a good idea. This always comes with a cost. Thats also the reason why basic data types (array, dictionaries, etc) are generally not thread safe… because its usually not needed or handled on a much higher level. Its a different story for languages/environments that are supposed to be safe by default and where you have language features that ensure safety (actors, optionals etc) but not for something like libc which has a standard it has to conform to and like 100 years of history.
- dgrunwald 2y ago
  The problem with `setenv` is that people expect one process to have one set of environment variables, which is shared across multiple languages running in that process. This implies every language must let its environment variables be managed by a central language-independent library -- and on POSIX systems, that's libc. So if libc refuses to provide thread-safety, that impacts not just C, but all possible languages (except for those that cannot call into C-libraries; as those don't need to bother synchronizing the environment with libc).
  - PaulDavisThe1st 2y ago
    It's not just that "libc refuses to provide thread-safety" ... the POSIX standard specifies that these functions are non-reentrant.
    - saagarjha 2y ago
      A conformant implementation can make a non-reentrant function actually safe under the hood for people that call into it erroneously. Unfortunately, there is no way to do this for getenv/setenv, because of the API they expose (specifically, when environ is accessed directly).
- tsimionescu 2y ago
  In some cases this is true. In the case of setting and getting env vars, it is not. There is no comceivable reason for making a process that spends any significant portion of its runtime calling setenv() or getenv(). Even if those calls were a thousand times slower than today, it would still be a non-issue.
cogman10 2y ago
You can't. You could wrap setenv in a mutex, but that's not good enough. It can still be called from different processes, which means you'd need to do a more expensive and complex syncing system to make it safe. That ballons out to other env related methods needing to honor the synchronization primitive in order for there to be a semblance of safety. However, you still end up in a scenario where you can call setenv getenv and that would be incorrect because between the set and the get, even with mutexes properly in place and coordinated amongst different applications, you have a race condition where your set can be overwritten by another application's set before your get can run. Now, instead of actually making these functions safe you've buried the fact that external processes (or your own threads) can mess with env state. The solution is to stop using env as some sort of global variable and instead treat it as a constant when the application starts. Using setenv should be mostly discouraged because of these issues.
- ryao 2y ago
  How does an external process mess with env state? As far as I know, you pass the environment when doing the execvpe() and then you cannot touch it from outside of the process anymore.
  - jenadine 2y ago
    You're correct. Parent comment is inaccurate. The problem is that a different library in the same process can use getenv without locking (or without locking the same lock as your code)
- tsimionescu 2y ago
  Of course you can. Mutexes are system objects, so it's not a huge problem to sync across processes, if you really have to (is it really expected that one process can set env vars inside another process?). Making global state, especially state that has no reason to be modified or even read very often like the env, thread safe is a trivial issue, well studied and understood. Could an intern do it? Probably not. Could literally any maintainer of a standard C library? Easily. This is much more of a culture problem preventing such obvious flaws from being recognized as such. Side-note: your set-then-get example is a theoretical problem in search of a use case. Why would you ever want to concurrently set an env var and expect to be guaranteed to read that same value? And even if this is a real thing that applications really use, exposing a new function to sync anything on the env mutex is, again, trivial. So, if you really needed that, you could do lockenv setenv getenv unlockenv And problem solved.
PaulDavisThe1st 2y ago
One of the major differences between X Window and the win32 GUI APIs is that the windows one builds in thread safety, and it cannot be removed. This means that you pay the price of mutexes and the like (what the windows world likes to call "critical sections"), even if you have a single threaded GUI. X Window, on the other hand, decided to do nothing about threads at all, leaving it up to the application. 30 years after these decisions were made, most sensible people do single threaded GUIs anyway (that is, all calls to the windowing API come from a single thread, and all redraws occur synchronously with respect to that thread; this does not block the use of threads functioning as workers on behalf of the GUI, but they are not allowed to make windowing API calls themselves). Consequently, the overhead present in the win32 API is basically just dead-weight, there to make sure that "things are safe by default". There's a design lesson here for everyone, though precisely what it is will likely still be argued about.
- wmf 2y ago
  Yet 30 years later people are calling setenv()/getenv() from different threads even though "it is known" that it crashes. For whatever reason the lesson from GUIs doesn't apply here.
  - PaulDavisThe1st 2y ago
    Judging from a lot of the comments in this thread, the idea that there could even be parts of the *POSIX API* that are not thread-safe seems like an idea that hasn't even occured to a lot of (younger?) programmers ...
- lmm 2y ago
  > Consequently, the overhead present in the win32 API is basically just dead-weight, there to make sure that "things are safe by default". How much overhead is it though? IIRC uncontended mutexes are practically free, especially when they're only being used from a single thread. Our industry is way too eager to make things unsafe for the sake of marginal performance differences that are irrelevant for most use cases, IMO.

deleted 2y ago

[deleted]

jabl 2y ago

> A non-reentrant function cannot be thread safe. Actually, a non-reentrant function can be thread-safe. A common example of such a function in libc being malloc().

adrian_b 2y ago
By definition, a "reentrant function" is a function that may be invoked even when it has not returned yet from a previous invocation. So a non-reentrant function is a function that may not be invoked again between a previous invocation and returning from that invocation. When a function may be invoked from different threads, then it is certain that sometimes it will be invoked by a thread before returning from a previous invocation from a different thread. Therefore any function that may be invoked from different threads must be reentrant. Otherwise the behavior of the program is unpredictable. Reentrant functions may be required even in single-thread programs, when they may be invoked recursively, or they may be invoked by signal handlers. An implementation of "malloc" may be reentrant or it may be non-reentrant. Old "malloc" implementations were usually non-reentrant because they used global variables for managing the heap. Such "malloc" functions could not be used in multi-threaded programs. Modern "malloc" implementations are reentrant, either by using only thread-local storage or by using shared global variables to which some method for concurrent access is implemented, e.g. with mutual exclusion.
- tedunangst 2y ago
  Who has a signal safe malloc?
  - adrian_b 2y ago
    POSIX does not require malloc to be signal safe. Therefore I do not think that anyone has bothered to implement a signal-safe malloc, as this is likely to be complicated. Allocating memory in a signal handler makes no sense in a well designed program, so not being allowed to use malloc and related functions is not a problem.
    - 2y ago