5 ms·

This is somewhat outdated. GNU make now uses named pipes by default on platforms that support it. https://www.gnu.org/software/make/manual/html_node/POSIX-Jobs

This is somewhat outdated. GNU make now uses named pipes by default on platforms that support it. https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html https://www.gnu.org/software/make/manual/html_node/POSIX-Job... It's a bit more robust. For example when you have a Makefile like this: target: +python3 script.py $@ where `script.py` itself spawns a sub-process that is jobserver aware (like make, gcc, cargo, ...), you don't have to worry about whether the fds of the python process, needed for communicating with the jobserver, are inherited in the subprocess. Previously you had to run `subprocess.Popen(..., close_fds=False)` and make sure MAKEFLAGS was inherited. Now you only have to ensure MAKEFLAGS is propagated, and that the sub-process can access the fifo by path (which is sometimes problematic, for example when running docker).

xyzzy_plugh 2y ago
Frustratingly there is still value in the old style jobserver because old versions of GNU make are prevalent in the wild, i.e. on macOS. I'm at the point where if you're going to require users to install something, rather than a newer version of make, I'd rather require a more modern build tool and disregard make entirely.
o11c 2y ago
Any program that closes FDs it doesn't know the origin of is fundamentally broken, and should have bugs filed against it. If "close all FDs" is a workaround for other bugs, fix those bugs too.
- int_19h 2y ago
  This particular argument is about closing inherited FDs in a forked child process. This is a perfectly sensible thing to do if the child doesn't need them, and most child processes do not need FDs for various random files opened by the parent.
  - o11c 2y ago
    "Random files opened by the parent" were, as a rule, deliberately given to the child. You would use CLOEXEC if you don't want that. Please don't let your children trash something you gave them just because they don't appreciate it yet.
    - int_19h 2y ago
      That's the way it's supposed to be, in the theory. But because this API requires extra effort to use, and by default FDs are inherited, in practice most of the time it is not intentional. The correct design would be to explicitly whitelist the FDs that the child process will inherit. This is exactly what Python does - its standard API to spawn subprocesses not only has close_fds=True set by default, it also has pass_fds where you list the ones that should not be closed. FWIW I also don't see why the parent should care if the child closes its inherited descriptors, since it retains its own copy which remains open.
      - themk 2y ago
        The problem is that process A might spawn process B that might spawn process C. A may have opened an fd for C to use, and B has no way of knowing that. You really shouldn't be closing random fds that you inherited.
- stabbles 2y ago
  It's the default in Python :)
- crest 2y ago
  Sure but the nice thing about a named pipe is that you can, but don't have to get file descriptor (through inheritance or fd passing). Instead you can e.g. put the path to it into an environment variable. Sometimes that's wanted sometimes not shrug.

xyzzy_plugh 2y ago

Frustratingly there is still value in the old style jobserver because old versions of GNU make are prevalent in the wild, i.e. on macOS. I'm at the point where if you're going to require users to install something, rather than a newer version of make, I'd rather require a more modern build tool and disregard make entirely.

o11c 2y ago

Any program that closes FDs it doesn't know the origin of is fundamentally broken, and should have bugs filed against it. If "close all FDs" is a workaround for other bugs, fix those bugs too.

int_19h 2y ago
This particular argument is about closing inherited FDs in a forked child process. This is a perfectly sensible thing to do if the child doesn't need them, and most child processes do not need FDs for various random files opened by the parent.
- o11c 2y ago
  "Random files opened by the parent" were, as a rule, deliberately given to the child. You would use CLOEXEC if you don't want that. Please don't let your children trash something you gave them just because they don't appreciate it yet.
  - int_19h 2y ago
    That's the way it's supposed to be, in the theory. But because this API requires extra effort to use, and by default FDs are inherited, in practice most of the time it is not intentional. The correct design would be to explicitly whitelist the FDs that the child process will inherit. This is exactly what Python does - its standard API to spawn subprocesses not only has close_fds=True set by default, it also has pass_fds where you list the ones that should not be closed. FWIW I also don't see why the parent should care if the child closes its inherited descriptors, since it retains its own copy which remains open.
    - themk 2y ago
      The problem is that process A might spawn process B that might spawn process C. A may have opened an fd for C to use, and B has no way of knowing that. You really shouldn't be closing random fds that you inherited.
stabbles 2y ago
It's the default in Python :)
crest 2y ago
Sure but the nice thing about a named pipe is that you can, but don't have to get file descriptor (through inheritance or fd passing). Instead you can e.g. put the path to it into an environment variable. Sometimes that's wanted sometimes not shrug.