7 ms·

Even waiting for a process to exit is surprisingly hard (impossible?) in Linux, unless it's your child.

wumms 7y ago
wait [-fn] [jobspec or pid …] https://www.gnu.org/software/bash/manual/html_node/Job-Control-Builtins.html https://www.gnu.org/software/bash/manual/html_node/Job-Contr...
- megous 7y ago
  "Wait until the child process..." So, no. That's not it.
megous 7y ago
Not anymore: b53b0b9d9a613c418057f6cb921c2f40a6f78c24 (pidfd: add polling support)
- mehrdadn 7y ago
  Wow, finally! Committed just 3 months ago. Might be able to finally rely on it in Linux distros a few years from now! https://github.com/torvalds/linux/commit/b53b0b9d9a613c418057f6cb921c2f40a6f78c24 https://github.com/torvalds/linux/commit/b53b0b9d9a613c41805...
  - pas 7y ago
    It's in kernel v5.3, which is expected in September, so it might be in Ubuntu Eoan Ermine 19.10. (After all 5.0 got released in March and Ubuntu 19.04 ships with 5.0.)
    - mehrdadn 7y ago
      Yeah, that's when you'll just start seeing the new kernel in the wild. Not when you'll stop seeing older kernels in the wild.
      - pas 7y ago
        There will be RHEL6 installs for the next 10 years, but why do they matter exactly?
AnssiH 7y ago
That is getting considerably easier with the addition of pidfds, though: https://lwn.net/SubscriberLink/794707/93ffb35438fd3710/ https://lwn.net/SubscriberLink/794707/93ffb35438fd3710/
- mehrdadn 7y ago
  Posted 1 week ago! Do you know what took them so long to realize this is the correct way to do it? Windows has had it for some two decades...
  - caspper69 7y ago
    Unfortunately, as Linux has matured, many of its developers refuse to adapt technologies for the sole reason that it "stinks" of Microsoft. Cutler and his team were pretty forward thinking given that NT is over 30 years old. See also: the async i/o headaches and the inability to WaitForSingleObject()/WaitForMultipleObjects() (or an analogue) in Linux. It's a shame really.
    - pas 7y ago
      re: WaitForMultipleObjects, finally something like that is likely coming to Linux: https://www.reddit.com/r/linux/comments/ck77gm/valve_proposes_kernel_changes_to_help_gaming https://www.reddit.com/r/linux/comments/ck77gm/valve_propose... > It's a shame really. Well, yes, and of course, yet at the same time it's FOSS, so ... if someone really needed it, they should have proposed a patch. ¯\_(ツ)_/¯
      - blattimwind 7y ago
        > Well, yes, and of course, yet at the same time it's FOSS, so ... if someone really needed it, they should have proposed a patch. ¯\_(ツ)_/¯ It's a bit like doing I/O that doesn't throw your data away... just use O_DIRECT, bring your own I/O manager and page cache and scheduling and simply implement all the low-level compatibility stuff yourself if you need such a weird thing ¯\_(ツ)_/¯
tus88 7y ago
Is it any easier on any other platform? Genuinely asking.
- tatersolid 7y ago
  Windows NT has had “kill process tree” since as far back as I recall (mid 1990s). See: taskkill /t
- tedunangst 7y ago
  kqueue allows waiting for any pid to exit.
mktmkr 7y ago
Isn’t netlink’s PROC_EVENT_EXIT pretty straightforward?
- cyphar 7y ago
  The proc connector has a few problems: 1. It's effectively unmaintained (which I found out when I started mailing around asking if there was interest in me sending patches that fix the rest of the issues listed). 2. It requires privileges to use, making it useless when compared to other alternatives that can help solve some of the other issues (pidfds or just the good old /proc/$pid fstatat(2) trick). 3. It doesn't work in containers at all. 4. It has several pretty serious bugs which could even be argued to be security bugs. But since it has effectively zero users now, I'd be surprised if anyone would be interested in such bugs. I wanted to fix these issues quite desperately, because it would allow for init systems that don't suffer from the cgroup or ptrace downsides. Unfortunately, it uses netlink and so any changes are mind-bogglingly complicated (especially if you want to tie it to PID namespaces because then you're really SoL since netlink is fundamentally tied to network namespaces).
- geofft 7y ago
  If you're a privileged process. Usually I, a non-root user, want to wait for another process I started in another process tree.
cyphar 7y ago
You could do it for a very long time. Folks are mentioning the new pidfd stuff, but that interface is built on much older tricks. In particular, what you could do is grab a handle to /proc/$pid. This is now called a pidfd, but this works on old kernels too. Then, to check if the process has died you just do a fstatat(2) and see if you get ESRCH -- if you do, the process has died and this will work even if the pid is reused. I think you could use inotify to avoid polling, but I'm not sure. The main benefit for the new poll support for pidfds is that you can get the exit status. And obviously CLONE_PIDFD has other benefits as well as the incredibly useful feature of pidfd_send_signal(2) which was the first patch sent for Linux 5.1.

wait [-fn] [jobspec or pid …] https://www.gnu.org/software/bash/manual/html_node/Job-Control-Builtins.html https://www.gnu.org/software/bash/manual/html_node/Job-Contr...

megous 7y ago
"Wait until the child process..." So, no. That's not it.

megous 7y ago

Not anymore: b53b0b9d9a613c418057f6cb921c2f40a6f78c24 (pidfd: add polling support)

mehrdadn 7y ago
Wow, finally! Committed just 3 months ago. Might be able to finally rely on it in Linux distros a few years from now! https://github.com/torvalds/linux/commit/b53b0b9d9a613c418057f6cb921c2f40a6f78c24 https://github.com/torvalds/linux/commit/b53b0b9d9a613c41805...
- pas 7y ago
  It's in kernel v5.3, which is expected in September, so it might be in Ubuntu Eoan Ermine 19.10. (After all 5.0 got released in March and Ubuntu 19.04 ships with 5.0.)
  - mehrdadn 7y ago
    Yeah, that's when you'll just start seeing the new kernel in the wild. Not when you'll stop seeing older kernels in the wild.
    - pas 7y ago
      There will be RHEL6 installs for the next 10 years, but why do they matter exactly?

AnssiH 7y ago

That is getting considerably easier with the addition of pidfds, though: https://lwn.net/SubscriberLink/794707/93ffb35438fd3710/ https://lwn.net/SubscriberLink/794707/93ffb35438fd3710/

mehrdadn 7y ago
Posted 1 week ago! Do you know what took them so long to realize this is the correct way to do it? Windows has had it for some two decades...
- caspper69 7y ago
  Unfortunately, as Linux has matured, many of its developers refuse to adapt technologies for the sole reason that it "stinks" of Microsoft. Cutler and his team were pretty forward thinking given that NT is over 30 years old. See also: the async i/o headaches and the inability to WaitForSingleObject()/WaitForMultipleObjects() (or an analogue) in Linux. It's a shame really.
  - pas 7y ago
    re: WaitForMultipleObjects, finally something like that is likely coming to Linux: https://www.reddit.com/r/linux/comments/ck77gm/valve_proposes_kernel_changes_to_help_gaming https://www.reddit.com/r/linux/comments/ck77gm/valve_propose... > It's a shame really. Well, yes, and of course, yet at the same time it's FOSS, so ... if someone really needed it, they should have proposed a patch. ¯\_(ツ)_/¯
    - blattimwind 7y ago
      > Well, yes, and of course, yet at the same time it's FOSS, so ... if someone really needed it, they should have proposed a patch. ¯\_(ツ)_/¯ It's a bit like doing I/O that doesn't throw your data away... just use O_DIRECT, bring your own I/O manager and page cache and scheduling and simply implement all the low-level compatibility stuff yourself if you need such a weird thing ¯\_(ツ)_/¯

tus88 7y ago

Is it any easier on any other platform? Genuinely asking.

tatersolid 7y ago
Windows NT has had “kill process tree” since as far back as I recall (mid 1990s). See: taskkill /t
tedunangst 7y ago
kqueue allows waiting for any pid to exit.

mktmkr 7y ago

Isn’t netlink’s PROC_EVENT_EXIT pretty straightforward?

cyphar 7y ago
The proc connector has a few problems: 1. It's effectively unmaintained (which I found out when I started mailing around asking if there was interest in me sending patches that fix the rest of the issues listed). 2. It requires privileges to use, making it useless when compared to other alternatives that can help solve some of the other issues (pidfds or just the good old /proc/$pid fstatat(2) trick). 3. It doesn't work in containers at all. 4. It has several pretty serious bugs which could even be argued to be security bugs. But since it has effectively zero users now, I'd be surprised if anyone would be interested in such bugs. I wanted to fix these issues quite desperately, because it would allow for init systems that don't suffer from the cgroup or ptrace downsides. Unfortunately, it uses netlink and so any changes are mind-bogglingly complicated (especially if you want to tie it to PID namespaces because then you're really SoL since netlink is fundamentally tied to network namespaces).
geofft 7y ago
If you're a privileged process. Usually I, a non-root user, want to wait for another process I started in another process tree.

cyphar 7y ago

You could do it for a very long time. Folks are mentioning the new pidfd stuff, but that interface is built on much older tricks. In particular, what you could do is grab a handle to /proc/$pid. This is now called a pidfd, but this works on old kernels too. Then, to check if the process has died you just do a fstatat(2) and see if you get ESRCH -- if you do, the process has died and this will work even if the pid is reused. I think you could use inotify to avoid polling, but I'm not sure. The main benefit for the new poll support for pidfds is that you can get the exit status. And obviously CLONE_PIDFD has other benefits as well as the incredibly useful feature of pidfd_send_signal(2) which was the first patch sent for Linux 5.1.