5 ms·

The point is, did you ever have to debug systemd because it was systemd that was broken and not other parts?

saurik 6y ago
Well, I would argue "the point is" the ability to easily understand when and how something works is useful; and, to that end, having the behaviors of something directly in front of you is thereby valuable in a way that shouldn't be dismissed, particularly when the language it is all written in is the "lineup franca" of system administration: shell. I have often wondered when something happens and why it happens at some point and simply grepped my way through /etc seeing exactly how the extremely simple startup behavior of those "cumbersome shell scripts" worked, and taking that away and hiding those behaviors into some complex C program to save a tiny bit of startup performance seems like a sad tradeoff, particularly for people just beginning their journey of discovery (like, I ostensibly don't need it anymore, as I can now read complex C programs and have in fact carefully scrutinized the code for the various eras of Apple's launchd, and so have a pretty solid understanding of exactly how everything works... but 25 years ago, being able to do that on Slackware because it effectively just had this beautifully simple shell script that just ran stuff as if it wasn't hard... that showed me the beauty of this system and how easy it is to bootstrap these things: the world didn't actually get more complex since then in a way where any of this cruft is actually necessary).
- kaba0 6y ago
  > to save a tiny bit of startup performance That is absolutely not the reason for systemd to exist, it is an additional benefit. How was early-boot logging, error handling, dependency handling between services implemented before? At best only in some ad-hoc way, different on each distro. The domain has an inherent complexity to it, init systems are not easy, there is no saving that. But I think systemd exposing the configurable parts in a declarative way is a net win over the previous solutions.
  - guenthert 6y ago
    >> to save a tiny bit of startup performance > That is absolutely not the reason for systemd to exist If you read the original web document announcing systemd, one could come away with the impression that it was though (even if substantiated with quite questionable performance tests using virtual machines).
    - kaba0 6y ago
      According to a core developer, it was a side effect of doing things the right way: http://0pointer.de/blog/projects/the-biggest-myths.html http://0pointer.de/blog/projects/the-biggest-myths.html
- saurik 6y ago
  lineup -> lingua :(
lmm 6y ago
When systemd came out I started experiencing "I upgraded and now my system won't boot". When I got fed up enough and removed systemd (and eventually had to switch distributions as non-systemd became unsupported) that stopped happening. I'm sure there will be some sophisticated argument about how this wasn't really systemd's fault and was someone else's fault, but the correlation is pretty damn strong.
ragnese 6y ago
That's a fair point, but where is the line? Do sysadmins worry about debugging Linux? Should it be written in Bash so that they can (obviously the kernel can't be written in Bash, but for the sake of argument)? As a non-sysadmin, I just feel like it's totally fair to have PID 1 be about as trusted and uniform as the kernel. If there's a bug in systemd, that's bad. If there's a bug in Linux, that's bad. But can't we at least aim for an init that is not just a bunch of Bash scripts running as root?
- drran 6y ago
  I have more trust in bash than in C. I can enable tracing in bash script and then see how it executed exactly. IMHO, we should aim for an open standard for the init subsystem instead. For example, I want to implement my init system in Rust, but systemd is blob, which is hard to replace.
  - ragnese 6y ago
    > I have more trust in bash than in C. I can enable tracing in bash script and then see how it executed exactly. If I had to work in either day-to-day, I'd chew my hands off. :p > IMHO, we should aim for an open standard for the init subsystem instead. For example, I want to implement my init system in Rust, but systemd is blob, which is hard to replace. Well, hold on. The person I replied to was talking about being able to debug systemd. So this is very tangential to that. If we had an standard for inits, with different implementations, it would be hard to debug all of the ones that are compiled binaries. My point is still that a sysadmin should not be expected to debug PID 1 any more than they should be expected to debug a driver. Also, you can just treat systemd's APIs as the standard. GNOME, for example, depends on launchd. And, IIRC, there was some shim created so that GNOME could run on a BSD, which obviously doesn't have systemd. So, go ahead and write your init in Rust- Rust has great C API/ABI compatibility!
    - drran 6y ago
      Minimal initd is very short: https://github.com/mesalock-linux/minit/blob/master/src/main.rs https://github.com/mesalock-linux/minit/blob/master/src/main... http://git.suckless.org/sinit/file/sinit.c.html http://git.suckless.org/sinit/file/sinit.c.html However, systemd is huge: 1.3 million lines of code. https://www.theregister.com/2020/01/06/linux_2020_kernel_systemd_code/ https://www.theregister.com/2020/01/06/linux_2020_kernel_sys...
guenthert 6y ago
Not the application itself, no, but its configuration. Once I managed to screw up things by modifying the fstab: I forgot the (never used) fs_freq and fs_passno fields in a new entry for /tmp. mountall et. al. are fault tolerant and mount the fs regardless, but not so systemd -- it decided best course of action would be to not mount any local fs. You can imagine debugging that was a wee bit hairy, particularly since it was a head-less system.

Well, I would argue "the point is" the ability to easily understand when and how something works is useful; and, to that end, having the behaviors of something directly in front of you is thereby valuable in a way that shouldn't be dismissed, particularly when the language it is all written in is the "lineup franca" of system administration: shell. I have often wondered when something happens and why it happens at some point and simply grepped my way through /etc seeing exactly how the extremely simple startup behavior of those "cumbersome shell scripts" worked, and taking that away and hiding those behaviors into some complex C program to save a tiny bit of startup performance seems like a sad tradeoff, particularly for people just beginning their journey of discovery (like, I ostensibly don't need it anymore, as I can now read complex C programs and have in fact carefully scrutinized the code for the various eras of Apple's launchd, and so have a pretty solid understanding of exactly how everything works... but 25 years ago, being able to do that on Slackware because it effectively just had this beautifully simple shell script that just ran stuff as if it wasn't hard... that showed me the beauty of this system and how easy it is to bootstrap these things: the world didn't actually get more complex since then in a way where any of this cruft is actually necessary).

kaba0 6y ago
> to save a tiny bit of startup performance That is absolutely not the reason for systemd to exist, it is an additional benefit. How was early-boot logging, error handling, dependency handling between services implemented before? At best only in some ad-hoc way, different on each distro. The domain has an inherent complexity to it, init systems are not easy, there is no saving that. But I think systemd exposing the configurable parts in a declarative way is a net win over the previous solutions.
- guenthert 6y ago
  >> to save a tiny bit of startup performance > That is absolutely not the reason for systemd to exist If you read the original web document announcing systemd, one could come away with the impression that it was though (even if substantiated with quite questionable performance tests using virtual machines).
  - kaba0 6y ago
    According to a core developer, it was a side effect of doing things the right way: http://0pointer.de/blog/projects/the-biggest-myths.html http://0pointer.de/blog/projects/the-biggest-myths.html
saurik 6y ago
lineup -> lingua :(

lmm 6y ago

When systemd came out I started experiencing "I upgraded and now my system won't boot". When I got fed up enough and removed systemd (and eventually had to switch distributions as non-systemd became unsupported) that stopped happening. I'm sure there will be some sophisticated argument about how this wasn't really systemd's fault and was someone else's fault, but the correlation is pretty damn strong.

ragnese 6y ago

That's a fair point, but where is the line? Do sysadmins worry about debugging Linux? Should it be written in Bash so that they can (obviously the kernel can't be written in Bash, but for the sake of argument)? As a non-sysadmin, I just feel like it's totally fair to have PID 1 be about as trusted and uniform as the kernel. If there's a bug in systemd, that's bad. If there's a bug in Linux, that's bad. But can't we at least aim for an init that is not just a bunch of Bash scripts running as root?

drran 6y ago
I have more trust in bash than in C. I can enable tracing in bash script and then see how it executed exactly. IMHO, we should aim for an open standard for the init subsystem instead. For example, I want to implement my init system in Rust, but systemd is blob, which is hard to replace.
- ragnese 6y ago
  > I have more trust in bash than in C. I can enable tracing in bash script and then see how it executed exactly. If I had to work in either day-to-day, I'd chew my hands off. :p > IMHO, we should aim for an open standard for the init subsystem instead. For example, I want to implement my init system in Rust, but systemd is blob, which is hard to replace. Well, hold on. The person I replied to was talking about being able to debug systemd. So this is very tangential to that. If we had an standard for inits, with different implementations, it would be hard to debug all of the ones that are compiled binaries. My point is still that a sysadmin should not be expected to debug PID 1 any more than they should be expected to debug a driver. Also, you can just treat systemd's APIs as the standard. GNOME, for example, depends on launchd. And, IIRC, there was some shim created so that GNOME could run on a BSD, which obviously doesn't have systemd. So, go ahead and write your init in Rust- Rust has great C API/ABI compatibility!
  - drran 6y ago
    Minimal initd is very short: https://github.com/mesalock-linux/minit/blob/master/src/main.rs https://github.com/mesalock-linux/minit/blob/master/src/main... http://git.suckless.org/sinit/file/sinit.c.html http://git.suckless.org/sinit/file/sinit.c.html However, systemd is huge: 1.3 million lines of code. https://www.theregister.com/2020/01/06/linux_2020_kernel_systemd_code/ https://www.theregister.com/2020/01/06/linux_2020_kernel_sys...

guenthert 6y ago

Not the application itself, no, but its configuration. Once I managed to screw up things by modifying the fstab: I forgot the (never used) fs_freq and fs_passno fields in a new entry for /tmp. mountall et. al. are fault tolerant and mount the fs regardless, but not so systemd -- it decided best course of action would be to not mount any local fs. You can imagine debugging that was a wee bit hairy, particularly since it was a head-less system.