6 ms·

Wait how does a user space daemon exhausting its stack lead to a kernel panic?

by unmole 5y ago

monocasa 5y ago
PID 1 is special in Unix systems. As the parent of all other processes (and child to none) it's not clear to the kernel what should happen when it exits.
comex 5y ago
Because the kernel intentionally panics [1] if the init process would otherwise exit in any way – whether because it called exit(), it was killed, or, in this case, it crashed. This is likely because Unix semantics treat the init process specially: any process whose parent dies is re-parented to the init process. It's not clear what should happen to these processes if the init process itself went away, so the kernel just gives up. [1] https://elixir.bootlin.com/linux/latest/source/kernel/exit.c#L794 https://elixir.bootlin.com/linux/latest/source/kernel/exit.c...
- Denvercoder9 5y ago
  I wonder if just restarting PID1 could be viable alternative?
  - jerf 5y ago
    It's generally dangerous to restart PID1 in an enviroment where, by definition, something happened to PID1 that it wasn't expecting. The state of the system is now unreliable, and it's exactly the sort of unreliable in exactly the right place that tends to lead to whopping security issues. Far too easy to end up with "Crash PID1 with 'blah blah blah', then when it restarts it ends up doing bad things X & Y".
    - cameronh90 5y ago
      Perhaps a distinction here can be made between running as a server OS and a desktop OS. In a server I generally want a crash immediately. But on a desktop I'd rather it limp along and give me a chance to finish writing my Hackernews post.
      - londons_explore 5y ago
        On a server I'd usually want it to limp along too... Better fire an alert but keep happy customers than cause a massive outage just because of someone's overly strict checkfail... It depends really what your server does, and what the consequences of it doing the wrong thing are.
        michaelmrose 5y ago
        Or just don't put stuff that crashes in pid1
        afc 5y ago
        Yeah, great idea: rather than worry about how to deal with software bugs, just never have bugs...
        michaelmrose 5y ago
        runit.c is 330 lines of code Keeping it small and simple to minimize bugs is perfectly viable and reasonable.
        a1369209993 5y ago
        Exactly. This is why systemd is a terrible design. The fastest and most secure code, that never crashes and never needs patching, is code that doesn't exist.
        modshatereality 5y ago
        seriously, if you're doing allocations in pid1 you fucked up.
        bigiain 5y ago
        If the consequences of one server wedging itself is a "massive outage" and "unhappy customers", then you probably don't really care about that outage or those customers. If you don't have enough redundancy and alerting and automated disaster recovery to keep your customer facing shit up when one server panics, you're just relying on luck to keep your customers happy. Fire an alert, remove that server from the load balancer, and fix the problem without your customers even noticing. Or make sure if you're running a hobby-project architected platform that your customer expectations and SLAs are clear up front, and let it go down until Monday morning when you'll get around to fixing it.
  - dirkt 5y ago
    A better alternative would be to keep PID1 as simple as possible, and do anything more complex in a subprocess. Systemd of course goes in the opposite direction: It assimilates as much functionality as possible from the OS into systemd (though to be fair, not all into PID1).
paulddraper 5y ago
Because it's PID 1.
mjevans 5y ago
Poorly designed security architecture and division of labor. A more idealized init / systemd would have all of the execution flow of PID 1 mathematically provably correct, and correspondingly have as small a footprint as possible there. All additional functions would run under one or more child processes (where the bulk of systemd would execute).
- bigiain 5y ago
  "Lets put the graphics drivers in ring 0, for better performance!" -- Windows NT architects, 1996 "Ummm, lets not do that, it's not such a great idea..." -- Windows Vista team, 2006
  - Spivak 5y ago
    Different trade offs for different eras and different constraints.
  - fomine3 5y ago
    Initial Windows NT uses user mode graphic driver, then NT 4.0 move it kernel mode. https://docs.microsoft.com/en-us/previous-versions//cc750820(v=technet.10) https://docs.microsoft.com/en-us/previous-versions//cc750820...
    - bigiain 5y ago
      Yep. In about '96. (Well, it came out late '96, so I suppose a bunch of that was actually done in '95.)
- failwhaleshark 5y ago
  Systemd is a swiss-army-kitchen-sink-knife monolith of brittle complexity. A proper init system similar to runit or s6 would be written in something safer (minimum unsafe) like Rust, be modular, simpler, follow UNIX philosophy, and not try to do everything in one process. Microkernel-style.

Wait how does a user space daemon exhausting its stack lead to a kernel panic?

by unmole 5y ago

Wait how does a user space daemon exhausting its stack lead to a kernel panic?

PID 1 is special in Unix systems. As the parent of all other processes (and child to none) it's not clear to the kernel what should happen when it exits.

comex 5y ago

Because the kernel intentionally panics [1] if the init process would otherwise exit in any way – whether because it called exit(), it was killed, or, in this case, it crashed. This is likely because Unix semantics treat the init process specially: any process whose parent dies is re-parented to the init process. It's not clear what should happen to these processes if the init process itself went away, so the kernel just gives up. [1] https://elixir.bootlin.com/linux/latest/source/kernel/exit.c#L794 https://elixir.bootlin.com/linux/latest/source/kernel/exit.c...

Denvercoder9 5y ago
I wonder if just restarting PID1 could be viable alternative?
- jerf 5y ago
  It's generally dangerous to restart PID1 in an enviroment where, by definition, something happened to PID1 that it wasn't expecting. The state of the system is now unreliable, and it's exactly the sort of unreliable in exactly the right place that tends to lead to whopping security issues. Far too easy to end up with "Crash PID1 with 'blah blah blah', then when it restarts it ends up doing bad things X & Y".
  - cameronh90 5y ago
    Perhaps a distinction here can be made between running as a server OS and a desktop OS. In a server I generally want a crash immediately. But on a desktop I'd rather it limp along and give me a chance to finish writing my Hackernews post.
    - londons_explore 5y ago
      On a server I'd usually want it to limp along too... Better fire an alert but keep happy customers than cause a massive outage just because of someone's overly strict checkfail... It depends really what your server does, and what the consequences of it doing the wrong thing are.
      - michaelmrose 5y ago
        Or just don't put stuff that crashes in pid1
        afc 5y ago
        Yeah, great idea: rather than worry about how to deal with software bugs, just never have bugs...
        michaelmrose 5y ago
        runit.c is 330 lines of code Keeping it small and simple to minimize bugs is perfectly viable and reasonable.
        a1369209993 5y ago
        Exactly. This is why systemd is a terrible design. The fastest and most secure code, that never crashes and never needs patching, is code that doesn't exist.
        modshatereality 5y ago
        seriously, if you're doing allocations in pid1 you fucked up.
      - bigiain 5y ago
        If the consequences of one server wedging itself is a "massive outage" and "unhappy customers", then you probably don't really care about that outage or those customers. If you don't have enough redundancy and alerting and automated disaster recovery to keep your customer facing shit up when one server panics, you're just relying on luck to keep your customers happy. Fire an alert, remove that server from the load balancer, and fix the problem without your customers even noticing. Or make sure if you're running a hobby-project architected platform that your customer expectations and SLAs are clear up front, and let it go down until Monday morning when you'll get around to fixing it.
- dirkt 5y ago
  A better alternative would be to keep PID1 as simple as possible, and do anything more complex in a subprocess. Systemd of course goes in the opposite direction: It assimilates as much functionality as possible from the OS into systemd (though to be fair, not all into PID1).

paulddraper 5y ago

Because it's PID 1.

mjevans 5y ago

Poorly designed security architecture and division of labor. A more idealized init / systemd would have all of the execution flow of PID 1 mathematically provably correct, and correspondingly have as small a footprint as possible there. All additional functions would run under one or more child processes (where the bulk of systemd would execute).

bigiain 5y ago
"Lets put the graphics drivers in ring 0, for better performance!" -- Windows NT architects, 1996 "Ummm, lets not do that, it's not such a great idea..." -- Windows Vista team, 2006
- Spivak 5y ago
  Different trade offs for different eras and different constraints.
- fomine3 5y ago
  Initial Windows NT uses user mode graphic driver, then NT 4.0 move it kernel mode. https://docs.microsoft.com/en-us/previous-versions//cc750820(v=technet.10) https://docs.microsoft.com/en-us/previous-versions//cc750820...
  - bigiain 5y ago
    Yep. In about '96. (Well, it came out late '96, so I suppose a bunch of that was actually done in '95.)
failwhaleshark 5y ago
Systemd is a swiss-army-kitchen-sink-knife monolith of brittle complexity. A proper init system similar to runit or s6 would be written in something safer (minimum unsafe) like Rust, be modular, simpler, follow UNIX philosophy, and not try to do everything in one process. Microkernel-style.