7 ms·

This is silly, setenv() isn't reentrant for the same reason that getopt() isn't reentrant: there's no valid reason to use it except at the very beginning of the

This is silly, setenv() isn't reentrant for the same reason that getopt() isn't reentrant: there's no valid reason to use it except at the very beginning of the program. The most common misuse I see is changing env before forking a child: nobody has to do that, execve() lets you pass arbitrary envp to the new process without changing yours. If you need to change env in threaded tests... frankly I think there was probably a better way to do whatever you're doing, but you can just declare a global lock and use it. I bet you could even LD_PRELOAD a custom setenv() that uses your lock. Nobody is pointing at concrete problems outside of Rust. Rust is just wrong here, sorry, the manpage has said this for a long time: > POSIX.1 does not require setenv() or unsetenv() to be reentrant. I think a more intellectually honest version of this article would have been "POSIX should have made setenv() reentrant", not "C is buggy": it's not buggy, it obviously complies with the standard. There's nothing to "fix", he wants to change the standard.

kibwen 3y ago
> the manpage has said this for a long time Nobody is disputing what the manpage says. What people are arguing is that the specification should be improved so as to no longer say that. Please stop quoting the Posix docs, as it merely broadcasts that one has missed the point here. Documenting the behavior does not automatically excuse the behavior. As for Rust, the Rust docs make it clear that the underlying mechanism is fraught. Rust is well-acquainted with trying to find satisfactory solutions to the unhelpful and nonsensical tech stack that has been foisted upon the world by decades of worse-is-better laziness. And even if Rust were to mark std::env::set_var as unsafe, that doesn't magically fix anything; the underlying mechanism is broken, and actually fixing it is beyond Rust's control. Only the platforms can fix it.
- jcalvinowens 3y ago
  > Documenting the behavior does not automatically excuse the behavior. It's a standard, and I'm citing the standard. The non-reentrant behavior doesn't need to be "excused", it is correct! If a microcontroller used a different bit than you expected it to for something, would the documentation need to be "excused" for "disagreeing" with you? That's how absurd what you're saying here sounds. > What people are arguing is that the specification should be improved That would be more reasonable, but that's not the argument. They're saying the standard is wrong. Standards can't be wrong, they're tautological. All that can be wrong is the programmer's understanding of them.
  - paulddraper 3y ago
    If your point is "the standard is correct as judged by the standard" your point is accurate and meaningless.
    - Joker_vD 3y ago
      Worse than that, it means that standards shouldn't be changed (e.g. to improve them): because the changed standard would be different from the standard and that would make it wrong by, well, being different from the standard! I've actually heard a similar argument about moral systems, about why the debater should not change his moral beliefs: it would be morally wrong for him to do that, you see, because then he'd then consider some things he currently considers morally right to be morally wrong and vice versa. Probably completely coincidentally, that debater was not a very nice person to talk to.
      - JohnFen 3y ago
        > it means that standards shouldn't be changed (e.g. to improve them) I think it's more nuanced than that. It means standards shouldn't be changed in a way that breaks backward compatibility. That seems like an important feature of any standard. Nobody is saying standards shouldn't be improved.
        Joker_vD 3y ago
        > They're saying the standard is wrong. Standards can't be wrong, they're tautological. All that can be wrong is the programmer's understanding of them. I personally do not possess enough of mental flexibility to believe simultaneously both that "the standard is not wrong, it can not possibly be wrong" and that "the standard should be improved". And how do you improve standards anyway, without breaking compatibility? Making a previously thread-safe function non-thread-safe is an incompatible change: the clients relied on the thread-safety but new implementations won't provide it. Making a previously non-thread-safe function thread-safe is an incompatible change: the clients would rely on the thread-safety but old implementations don't provide it. And adding a new function to the standard is an undue burden on the implementers, and for almost no value since no client uses that function yet, and they won't for years because they want to keep backward compatibility with existing implementations.
- Ferret7446 3y ago
  > What people are arguing is that the specification should be improved so as to no longer say that What I hear is "we should invalidate all environments in existence", for the purpose of, um, something. Satisfying some Rust devs, I guess?
loevborg 3y ago
You nailed it. The real villain in this story is mutability. We're addicted to changing variables in place, which is inherently complex – especially so in multithreaded environments. Environment variables are clearly best treated as immutable. Rust, despite its advances in some areas, perpetuates our addiction to mutable variables.
- orange_fritter 3y ago
  Every attempt at escaping mutability basically kills the language in the mainstream because so much of "real" programming is just bit-twiddling that gets too verbose when immutability is involved. It's a good question whether Rust nudges the world toward functional/declarative spiritual purity by placing constraints on mutation. I'm betting that No, it doesn't.
  - vacuity 3y ago
    An explicit goal of Rust is to be "low level", which is an admittedly vague phrase. While you could certainly write a Rust library that clones linked lists left and right, and maybe someone would prefer that, someone else should be able to write a library that does in-place mutation with that delicious imperative goodness (badness?). To GP's point about environment variables, I think that's more of an issue with the fact that Rust tries to be compatible with existing C conventions in the OS. I don't think Rust can do much about it, since it can interface with C libraries that don't care about any Rust constraints on environment variables.
beltsazar 3y ago
> This is silly, setenv() isn't reentrant for the same reason that getopt() isn't reentrant: there's no valid reason to use it except at the very beginning of the program. It is not, unless you'd argue that "there's no valid reason to use [anything that transitively uses setenv()] except at the very beginning of the program." Did you even read the article? The author and the GitHub links mentioned provide some examples that use setenv() not directly, but transitively.
- jcalvinowens 3y ago
  > Did you even read the article? Of course. I also clicked through and looked at his examples, did you? > The author and the GitHub links mentioned provide some examples that use setenv() not directly, but transitively. The big list at the end of the article? That's absolutely not true: they're all read only usecases that prove my point. Nobody should be changing any of those in the middle of the program. If you disagree, please point out specifically which one and explain why, because I don't see it.
  - beltsazar 3y ago
    Ah yes, silly mistakes—I mixed up getenv and setenv. Those examples transitively use getenv, not setenv. Having said that, my point still stands. You can only control what you directly use / don't use. Third-party libraries you use might use setenv at times other than "the very beginning of the program."
    - verall 3y ago
      In C/C++ land you control what third party libraries you use / don't use and you don't use the ones that behave badly
      - vacuity 3y ago
        Imagine if programmers did that with memory safety. "Hey, is the program interfacing with any C/C++ libraries that aren't rigorously sanitized and fuzzed for memory safety? If so, throw them out." Yes, theoretically you can control these things. In practice, most people aren't going to and the few that try may have to expend a lot of effort.
duped 3y ago
> there's no valid reason to use it except at the very beginning of the program. POSIX doesn't define the "beginning of a program." Nor do you, if you're compiling C code. Libraries can (and do) spawn threads before main, so it's not even safe to use if you restrict yourself to "only the beginning of the program." > The most common misuse I see is changing env before forking a child: nobody has to do that, execve() lets you pass arbitrary envp to the new process without changing yours. Just remember to do it before fork() and not between fork() and exec() because you probably want to copy the existing envp and allocators usually aren't async signal safe. If you want to be sure to be correct, use posix_spawn to create a child process. -- I think it's fair to say "setenv is buggy" in the sense that the POSIX specification for setenv guarantees it to be buggy in most programs that think they should be using it. What makes it an even bigger footgun is that the path of least resistance for people who need/want the behavior is the most difficult to use correctly. POSIX is full of shit like this, and "you should know better" isn't a good enough excuse. It's like saying "hey you should have known unless you press the doohickey for 5 seconds and turn the chainsaw at 90 degrees when you start it, the chain is going to fly off."
- jcalvinowens 3y ago
  > Libraries can (and do) spawn threads before main I don't think any library should be calling setenv(), there's always a better way. If you know of a counterexample, please share it, I'd like to see it. > Just remember to do it before fork() and not between fork() and exec() because you probably want to copy the existing envp and allocators usually aren't async signal safe. Why would you go to all that trouble? Most implementations just use the stack to build the arguments for execve(), making all that irrelevant. > If you want to be sure to be correct, use posix_spawn to create a child process. That's not the purpose of posix_spawn(), it exists to deal with vfork-only nommu architectures: >> [posix_spawn() was] specified by POSIX to provide a standardized method of creating new processes on machines that lack the capability to support the fork(2) system call. >> These functions are not meant to replace the fork(2) and execve(2) system calls. In fact, they provide only a subset of the functionality that can be achieved by using the system calls. Anyway... back to our regularly scheduled programming: > the POSIX specification for setenv guarantees it to be buggy in most programs that think they should be using it. [...] POSIX is full of shit like this, and "you should know better" isn't a good enough excuse. I completely disagree. The C standard library is chock full of non-reentrant APIs. Nobody who has read more than ten manpages would reasonably assume that anything is reentrant without an explicit assurance. Locks are trivial. POSIX expects you to use a lock if you want to use setenv() like this. I also want to point out that (at least on Linux) getenv() returns a pointer to the stack! How could anybody with basic programming literacy reasonably expect that to be thread safe? It's exactly analogous to getopt() and argv. No, the authors obviously couldn't imagine that $FAANG would be building 4GB binaries with 1000+ recursive library dependencies, each of which has its own chance to reenact the printer scene from Officespace with your shared envp. I think that's an organizational problem, not a POSIX problem. I'm not saying the standard shouldn't change: I would love to see argv and envp become immutable. If we're going to change it, that's the right move IMHO. But I don't really think that's practical... > It's like saying "hey you should have known unless you press the doohickey for 5 seconds and turn the chainsaw at 90 degrees when you start it, the chain is going to fly off." I think it's more like saying "we can't stop people from getting hurt jaywalking, so we're going to solve the problem by legally requiring everybody to wear helmets at all times outdoors".
zlg_codes 3y ago
That's essentially the problem with any technology, or community around said technology, that's primed against some target they want to replace. It has kept me away from Rust for years. If its fans weren't such fanboys for disruptive activity like rewriting things in Rust for the fuck of it, I might look closer into it. But considering where it came from and their politics, it doesn't seem like Rust is actually for everyone. Its angle of picking on C is also rich. If C is so bad, why has no major language supplanted it or C++? Anything with high importance on performance is written in low level languages, unconcerned with pushing a narrative or making BS 'more inclusive' which is really another view on affirmative action. My identity alone makes me unfit for the project.
- mike_hock 3y ago
  > If C is so bad, why has no major language supplanted it or C++? Because no one, not even Rust, aims for feature parity with C (except on the language level).
- jcalvinowens 3y ago
  > or community around said technology, that's primed against some target they want to replace. It's a vocal minority of the community, in my experience. Most people I've personally met who are passionate about Rust have a much more reasonable attitude about the whole thing, and see it as moving the needle in the right direction rather than a fully formed solution. It also really is interesting: obviously it's not the panacea it is sometimes made out to be, but it truly does eliminate a class of error. I admit to a bit of stubbornness myself, but I'm trying to work with it more.
- JohnFen 3y ago
  > It has kept me away from Rust for years. Yes. The Rust community is what has kept me away from Rust for a long, long time. Now I'm learning Rust because it may become an important skill to have, but it's despite the community. They're very hard to put up with.
- Georgelemental 3y ago
  > But considering where it came from and their politics > unconcerned with pushing a narrative or making BS 'more inclusive' which is really another view on affirmative action For what it's worth: I am conservative, right-wing, as opposed to "affirmative action" as it is possible to be—and I love Rust. Don't judge the language by the politics of a tiny section of its community, judge it on the technical merits.
boring_twenties 3y ago
> The most common misuse I see is changing env before forking a child: nobody has to do that, execve() lets you pass arbitrary envp to the new process without changing yours. That's pretty much never what you would want. You want to set a single variable while inheriting the rest of the existing environment. In order to do that with execve() you would have to copy the existing environment first, yuck. And you wouldn't use setenv() before forking anyway, you would do it after forking, in the child, before exec.
- Too 3y ago
  What is yuck about copying the existing environment first?
  - boring_twenties 3y ago
    It's a complete waste all around? Wasted cycles at runtime, wasted development time, and a bunch of unnecessary lines of code to read and debug later.
    - Too 3y ago
      Wasted lines is a syntax sugar problem. It can be solved in the language level. Python for example has dict destructuring where you can solve this in under 10 characters. You could also imagine wrapper functions to execv taking in modifications to env only, that it merges with the inherited env. Copying is something that happens anyway if you write to the environment after fork, due to the copy-on-write model used by forking. If you don’t need to edit the environment at all, one could imagine a pure inherit option used. Besides, what size is the environment variables anyway? There are many other bigger bottlenecks if you are launching processes frequent enough for this to matter.

kibwen 3y ago

> the manpage has said this for a long time Nobody is disputing what the manpage says. What people are arguing is that the specification should be improved so as to no longer say that. Please stop quoting the Posix docs, as it merely broadcasts that one has missed the point here. Documenting the behavior does not automatically excuse the behavior. As for Rust, the Rust docs make it clear that the underlying mechanism is fraught. Rust is well-acquainted with trying to find satisfactory solutions to the unhelpful and nonsensical tech stack that has been foisted upon the world by decades of worse-is-better laziness. And even if Rust were to mark std::env::set_var as unsafe, that doesn't magically fix anything; the underlying mechanism is broken, and actually fixing it is beyond Rust's control. Only the platforms can fix it.

jcalvinowens 3y ago
> Documenting the behavior does not automatically excuse the behavior. It's a standard, and I'm citing the standard. The non-reentrant behavior doesn't need to be "excused", it is correct! If a microcontroller used a different bit than you expected it to for something, would the documentation need to be "excused" for "disagreeing" with you? That's how absurd what you're saying here sounds. > What people are arguing is that the specification should be improved That would be more reasonable, but that's not the argument. They're saying the standard is wrong. Standards can't be wrong, they're tautological. All that can be wrong is the programmer's understanding of them.
- paulddraper 3y ago
  If your point is "the standard is correct as judged by the standard" your point is accurate and meaningless.
  - Joker_vD 3y ago
    Worse than that, it means that standards shouldn't be changed (e.g. to improve them): because the changed standard would be different from the standard and that would make it wrong by, well, being different from the standard! I've actually heard a similar argument about moral systems, about why the debater should not change his moral beliefs: it would be morally wrong for him to do that, you see, because then he'd then consider some things he currently considers morally right to be morally wrong and vice versa. Probably completely coincidentally, that debater was not a very nice person to talk to.
    - JohnFen 3y ago
      > it means that standards shouldn't be changed (e.g. to improve them) I think it's more nuanced than that. It means standards shouldn't be changed in a way that breaks backward compatibility. That seems like an important feature of any standard. Nobody is saying standards shouldn't be improved.
      - Joker_vD 3y ago
        > They're saying the standard is wrong. Standards can't be wrong, they're tautological. All that can be wrong is the programmer's understanding of them. I personally do not possess enough of mental flexibility to believe simultaneously both that "the standard is not wrong, it can not possibly be wrong" and that "the standard should be improved". And how do you improve standards anyway, without breaking compatibility? Making a previously thread-safe function non-thread-safe is an incompatible change: the clients relied on the thread-safety but new implementations won't provide it. Making a previously non-thread-safe function thread-safe is an incompatible change: the clients would rely on the thread-safety but old implementations don't provide it. And adding a new function to the standard is an undue burden on the implementers, and for almost no value since no client uses that function yet, and they won't for years because they want to keep backward compatibility with existing implementations.
Ferret7446 3y ago
> What people are arguing is that the specification should be improved so as to no longer say that What I hear is "we should invalidate all environments in existence", for the purpose of, um, something. Satisfying some Rust devs, I guess?

loevborg 3y ago

You nailed it. The real villain in this story is mutability. We're addicted to changing variables in place, which is inherently complex – especially so in multithreaded environments. Environment variables are clearly best treated as immutable. Rust, despite its advances in some areas, perpetuates our addiction to mutable variables.

orange_fritter 3y ago
Every attempt at escaping mutability basically kills the language in the mainstream because so much of "real" programming is just bit-twiddling that gets too verbose when immutability is involved. It's a good question whether Rust nudges the world toward functional/declarative spiritual purity by placing constraints on mutation. I'm betting that No, it doesn't.
- vacuity 3y ago
  An explicit goal of Rust is to be "low level", which is an admittedly vague phrase. While you could certainly write a Rust library that clones linked lists left and right, and maybe someone would prefer that, someone else should be able to write a library that does in-place mutation with that delicious imperative goodness (badness?). To GP's point about environment variables, I think that's more of an issue with the fact that Rust tries to be compatible with existing C conventions in the OS. I don't think Rust can do much about it, since it can interface with C libraries that don't care about any Rust constraints on environment variables.

beltsazar 3y ago

> This is silly, setenv() isn't reentrant for the same reason that getopt() isn't reentrant: there's no valid reason to use it except at the very beginning of the program. It is not, unless you'd argue that "there's no valid reason to use [anything that transitively uses setenv()] except at the very beginning of the program." Did you even read the article? The author and the GitHub links mentioned provide some examples that use setenv() not directly, but transitively.

jcalvinowens 3y ago
> Did you even read the article? Of course. I also clicked through and looked at his examples, did you? > The author and the GitHub links mentioned provide some examples that use setenv() not directly, but transitively. The big list at the end of the article? That's absolutely not true: they're all read only usecases that prove my point. Nobody should be changing any of those in the middle of the program. If you disagree, please point out specifically which one and explain why, because I don't see it.
- beltsazar 3y ago
  Ah yes, silly mistakes—I mixed up getenv and setenv. Those examples transitively use getenv, not setenv. Having said that, my point still stands. You can only control what you directly use / don't use. Third-party libraries you use might use setenv at times other than "the very beginning of the program."
  - verall 3y ago
    In C/C++ land you control what third party libraries you use / don't use and you don't use the ones that behave badly
    - vacuity 3y ago
      Imagine if programmers did that with memory safety. "Hey, is the program interfacing with any C/C++ libraries that aren't rigorously sanitized and fuzzed for memory safety? If so, throw them out." Yes, theoretically you can control these things. In practice, most people aren't going to and the few that try may have to expend a lot of effort.

duped 3y ago

> there's no valid reason to use it except at the very beginning of the program. POSIX doesn't define the "beginning of a program." Nor do you, if you're compiling C code. Libraries can (and do) spawn threads before main, so it's not even safe to use if you restrict yourself to "only the beginning of the program." > The most common misuse I see is changing env before forking a child: nobody has to do that, execve() lets you pass arbitrary envp to the new process without changing yours. Just remember to do it before fork() and not between fork() and exec() because you probably want to copy the existing envp and allocators usually aren't async signal safe. If you want to be sure to be correct, use posix_spawn to create a child process. -- I think it's fair to say "setenv is buggy" in the sense that the POSIX specification for setenv guarantees it to be buggy in most programs that think they should be using it. What makes it an even bigger footgun is that the path of least resistance for people who need/want the behavior is the most difficult to use correctly. POSIX is full of shit like this, and "you should know better" isn't a good enough excuse. It's like saying "hey you should have known unless you press the doohickey for 5 seconds and turn the chainsaw at 90 degrees when you start it, the chain is going to fly off."

jcalvinowens 3y ago
> Libraries can (and do) spawn threads before main I don't think any library should be calling setenv(), there's always a better way. If you know of a counterexample, please share it, I'd like to see it. > Just remember to do it before fork() and not between fork() and exec() because you probably want to copy the existing envp and allocators usually aren't async signal safe. Why would you go to all that trouble? Most implementations just use the stack to build the arguments for execve(), making all that irrelevant. > If you want to be sure to be correct, use posix_spawn to create a child process. That's not the purpose of posix_spawn(), it exists to deal with vfork-only nommu architectures: >> [posix_spawn() was] specified by POSIX to provide a standardized method of creating new processes on machines that lack the capability to support the fork(2) system call. >> These functions are not meant to replace the fork(2) and execve(2) system calls. In fact, they provide only a subset of the functionality that can be achieved by using the system calls. Anyway... back to our regularly scheduled programming: > the POSIX specification for setenv guarantees it to be buggy in most programs that think they should be using it. [...] POSIX is full of shit like this, and "you should know better" isn't a good enough excuse. I completely disagree. The C standard library is chock full of non-reentrant APIs. Nobody who has read more than ten manpages would reasonably assume that anything is reentrant without an explicit assurance. Locks are trivial. POSIX expects you to use a lock if you want to use setenv() like this. I also want to point out that (at least on Linux) getenv() returns a pointer to the stack! How could anybody with basic programming literacy reasonably expect that to be thread safe? It's exactly analogous to getopt() and argv. No, the authors obviously couldn't imagine that $FAANG would be building 4GB binaries with 1000+ recursive library dependencies, each of which has its own chance to reenact the printer scene from Officespace with your shared envp. I think that's an organizational problem, not a POSIX problem. I'm not saying the standard shouldn't change: I would love to see argv and envp become immutable. If we're going to change it, that's the right move IMHO. But I don't really think that's practical... > It's like saying "hey you should have known unless you press the doohickey for 5 seconds and turn the chainsaw at 90 degrees when you start it, the chain is going to fly off." I think it's more like saying "we can't stop people from getting hurt jaywalking, so we're going to solve the problem by legally requiring everybody to wear helmets at all times outdoors".

zlg_codes 3y ago

That's essentially the problem with any technology, or community around said technology, that's primed against some target they want to replace. It has kept me away from Rust for years. If its fans weren't such fanboys for disruptive activity like rewriting things in Rust for the fuck of it, I might look closer into it. But considering where it came from and their politics, it doesn't seem like Rust is actually for everyone. Its angle of picking on C is also rich. If C is so bad, why has no major language supplanted it or C++? Anything with high importance on performance is written in low level languages, unconcerned with pushing a narrative or making BS 'more inclusive' which is really another view on affirmative action. My identity alone makes me unfit for the project.

mike_hock 3y ago
> If C is so bad, why has no major language supplanted it or C++? Because no one, not even Rust, aims for feature parity with C (except on the language level).
jcalvinowens 3y ago
> or community around said technology, that's primed against some target they want to replace. It's a vocal minority of the community, in my experience. Most people I've personally met who are passionate about Rust have a much more reasonable attitude about the whole thing, and see it as moving the needle in the right direction rather than a fully formed solution. It also really is interesting: obviously it's not the panacea it is sometimes made out to be, but it truly does eliminate a class of error. I admit to a bit of stubbornness myself, but I'm trying to work with it more.
JohnFen 3y ago
> It has kept me away from Rust for years. Yes. The Rust community is what has kept me away from Rust for a long, long time. Now I'm learning Rust because it may become an important skill to have, but it's despite the community. They're very hard to put up with.
Georgelemental 3y ago
> But considering where it came from and their politics > unconcerned with pushing a narrative or making BS 'more inclusive' which is really another view on affirmative action For what it's worth: I am conservative, right-wing, as opposed to "affirmative action" as it is possible to be—and I love Rust. Don't judge the language by the politics of a tiny section of its community, judge it on the technical merits.

boring_twenties 3y ago

> The most common misuse I see is changing env before forking a child: nobody has to do that, execve() lets you pass arbitrary envp to the new process without changing yours. That's pretty much never what you would want. You want to set a single variable while inheriting the rest of the existing environment. In order to do that with execve() you would have to copy the existing environment first, yuck. And you wouldn't use setenv() before forking anyway, you would do it after forking, in the child, before exec.

Too 3y ago
What is yuck about copying the existing environment first?
- boring_twenties 3y ago
  It's a complete waste all around? Wasted cycles at runtime, wasted development time, and a bunch of unnecessary lines of code to read and debug later.
  - Too 3y ago
    Wasted lines is a syntax sugar problem. It can be solved in the language level. Python for example has dict destructuring where you can solve this in under 10 characters. You could also imagine wrapper functions to execv taking in modifications to env only, that it merges with the inherited env. Copying is something that happens anyway if you write to the environment after fork, due to the copy-on-write model used by forking. If you don’t need to edit the environment at all, one could imagine a pure inherit option used. Besides, what size is the environment variables anyway? There are many other bigger bottlenecks if you are launching processes frequent enough for this to matter.