6 ms·

I am old enough to remember that problems with systemd were less technical and more political. People didn't like the way systemd developers pushed the communi

I am old enough to remember that problems with systemd were less technical and more political. People didn't like the way systemd developers pushed the community to adopt systemd, specially when they asked for 3rd party developers to make systemd a hard dependency. Unfortunately,people don't remember this today, and think users resisted to systemd adoption only because they didn't like systemd.

goodpoint 5y ago
> enough to remember We are talking 10 years ago, not 1975 here. Unless everybody is 25 years old on HN...
- int_19h 5y ago
  Or they weren't using Linux 10 years ago?
jandrese 5y ago
SystemD is kind of like MacOS, if you have a typical use case then it makes everything very easy. If you are doing something weird it can make your life a living hell.
- Denvercoder9 5y ago
  > If you are doing something weird it can make your life a living hell. I find it much easier to reliably accomplish non-standard things with systemd unit definitions, than it used to be with the ill-defined complex set of shell scripts we had before.
  - jcrawfordor 5y ago
    This issue is a lot bigger outside of services... I think the systemd service architecture is a clear improvement over SysV. The bigger issues I run into with systemd flexibility are related to the many other aspects of it. systemd-resolved, for example, has a very "opinionated" (to be polite) set of expectations about the environment and the systemd project tends to view any complaints about it not working outside of that environment as being the fault of the complainant. This leads to, well, DNSSEC is basically just broken and split horizon DNS is extremely unreliable. The former is sad but mostly doesn't matter, the latter leads to a lot of corporate networks having to disable resolved (I think configuring it to remove the semi-hardcoded "backup" resolvers reliably fixes this problem but honestly I find it completely ridiculous that resolved has a hardcoded list of DNS servers it just uses instead sometimes. It's unclear to me whether or not that's a bug at this point and I got tired of trying to follow the issues and mailing list threads where the developers were, uh, not amazingly helpful). systemd management of mounts can be similarly narrow about the types of configurations it supports, and systemd-firewalld just sort of openly only claims it can support simple use-cases. systemd-journald is also a net loss of flexibility compared to rsyslog but, on the other hand, rsyslog could quickly turn into an inscrutable mess if you used any of the advanced features (rainerscript...), so this may not be an entirely bad thing. And in general systemd is, well, opinionated. I hate to bang on resolved too much but I just happen to have spent quite a few hours last week figuring out resolved problems. Resolved does not handle "dotless" domains correctly in a lot of existing environments (it's very particular about exactly how the search domain is set up). Poettering has basically responded that it's because dotless domains are stupid and no one should use them, so it won't be changed. I don't necessarily disagree that dotless domains are not a good idea today but it does mean that resolved breaks a lot of older corporate and institutional environments that have been using them successfully for decades. This manifests as "I updated my distro and the intranet stopped working." That kind of breaking change is not very common with core Linux services and isn't going to make many friends in the IT crowd.
    - mianos 5y ago
      I think systemd is OK for a lot of things, I don't even hate journald. The API pretty good. But, I am 100% agreement on the whole resolvd thing. It is a complete fiasco for anything but someone's idea of a standard network. Even the most basic things, like waiting for DNS to be up before mounting a network filesystem has to be done by writing your own units if you don't want to just try and mount and hope for the best.
    - crazy_hombre 5y ago
      There's no such thing as systemd-firewalld.
      - jcrawfordor 5y ago
        Yeah, sorry, I'm blaming more on Poettering than is really fair. But firewalld is closely coupled with the overall systemd architecture and the projects are interconnected.
    - panick21_ 5y ago
      Systemd-resolved has some good ideas but can be a bit confusing. Overall its not bad, but it is a bit strange.
- Gigachad 5y ago
  Can you give some examples?
  - jandrese 5y ago
    Trying to set up network booting with NFS mounted root with SystemD is a challenge to say the least.
  - hedora 5y ago
    Not the person you’re responding to, but try getting it to mount home directories over NFS. I’m sure there are many other examples. Honestly, even after 10 years, I still have’t had a single positive interaction with it vs. traditional init. From what I’ve seen, old-school init handled (and continues to handle) the problems systemd “solved” more robustly and elegantly. I don’t get it.
    - bonzini 5y ago
      I'm not sure what's hard about mounting home directories over NFS?
- gotar 5y ago
  On the contrary - everything in systemd is much easier than before. Consider a situation, where your sh-backed SysV init was missing some function you required. Something between start-stop-daemon() or ifup(). You could throw a bunch of shell commands (note: most shell "programmers" are not capable of doing it properly; they do some '==' comparisons in /bin/sh shebang, don't handle errors, ignore fact that variables might contain "funny" characters etc. - but in general most of the scripts I've seen during my 24 years in Linux was simply badly written, meaning at least using bashizms). And then ...you watch out for every single package update that would overwrite your changes. Or you could polish them and try to push upstream. Now the situation seems to be worse - you can't just edit some random file to add missing function, you need to write it in source C files and recompile. Not a way for quick and dirty solution. But instead, you can simply write the same (s)hell commands in a script and call it from appropriate unit. No messing with thousand-lines scripts, no risk of overwrites, no need for upstreaming.
imoverclocked 5y ago
Network configuration (on say, Debian) is still a pain and has a lot of different ways of accomplishing similar tasks. Anything beyond the plain vanilla singular network device that is either dhcp/static is often painful. Systemd has some sauce, there is legacy /etc/network* stuff, there are daemons which can be configured graphically… it’s still a mess. In this arena, I think systemd has failed for the last decade. It introduced yet another way of doing things without considering the whole scope of what it needed to replace. I’m old enough to remember the culture wars and even older to remember a well thought out approach to overall configuration in Debian given the available tools of the time.
- taftster 5y ago
  Yes, this. The problem with systemd isn't systemd. It's that it never really replaced the other "legacy" parts effectively. Networking is a fantastic example. I'd be OK if there was one-definitive-place to configure networking, but when I read an article or follow a stack-overflow post, I'm not sure if what I'm changing is the most correct change or if its deprecated in favor of something newer. Systemd, I thought, would help settle this problem, but instead has only contributed to it.
  - massysett 5y ago
    Isn’t the number one problem here that you’re following a Stack Overflow post? Your system should have good documentation on how to configure networking, or some obvious sanctioned way to do it, such as a GUI widget. If your system has neither, that’s the problem. If it has one or the other and you’re using Stack Overflow anyway, that’s the problem. Note I said “good” documentation - many Linux systems have lousy documentation and they lack an obvious, sanctioned way to do it. Again, that’s the problem.
- jasonjayr 5y ago
  Networking on linux is wildly powerful. Most desktop/SOHO installations would just be Wifi + DHCP. Maybe some VPN in the mix. But on the other side of the scale, there are routers with complex netfiltering, vlans, crazy GRE Tunnels, Tun/Tap, Wireguard, macvlans split between namespaces, bridges, etc, you name it. Defining a straightforward configuration structure for that complexity is going to be insane. NetworkManager + Debian's /etc/network/interfaces (IMHO) seems to be a good fit: auto config w/ a GUI in NM in the simple case, and if the interface is listed in /etc/network/interfaces, you can do it anyway you want with Debian's system, with pre-up, up, post-up, post-down, etc scripts to your heart's content to setup whatever complexity you need.
throwaway984393 5y ago
Part of it was political, but it was also just practical. Poettering's designs are some of the worst for usability I've ever seen. They technically work, but you need a million StackOverflow Q&As to figure out how to get anything done or solve the many common problems. It's like somebody decided that the problem with Linux was it wasn't complicated enough. And then everbody just went with it.
- erik_seaberg 5y ago
  I keep coming back to the https://news.ycombinator.com/item?id=19023885 https://news.ycombinator.com/item?id=19023885 issue. There’s a default setting somewhere that makes systemd knowingly kill normal user processes it did not start. Why does that exist? It was never init’s job, and init is not something end users should have to be aware of to run stuff with POSIX nohup.
  - throwaway984393 5y ago
    Jesus. I didn't know about that, but it totally makes sense. Like when systemd devs told Linus he should change his kernel to fix a bug in systemd.
fartcannon 5y ago
I still don't trust it, given how aggressively it was pushed.
mihcsab 5y ago
My problem is that things start to depend on it, like snapd... I was looking into running anbox on wsl today, but I realized I need to compile stuff, because they distribute anbox in snap form only, so I kind of gave up...
mariusor 5y ago
> the way systemd developers pushed the community to adopt systemd, specially when they asked for 3rd party developers to make systemd a hard dependency. Do you have any links for these statements? My memory is not great but I think I would remember anything similar to this.
pas 5y ago
> systemd developers pushed the community [citation needed] I remember spending days reading the arguments on the Debian mailing list. There was no push from systemd devs. (Endless threads about upstart, how tweaking upstart would solve everything and it's already in Ubuntu, and a lots of questions about guarantees. Are openrc/upstart/systemd maintainers willing to guarantee this or that? And as far as I remember, there were no guarantees, obviously.) Before that when the hwdb and udev merged into systemd, people got the pitchforks out, it was the decision by Kay, but obviously "systemd devs" were somehow behind it. When Greg KH supported kdbus, somehow it was again those meddling systemd devs. Somehow OSS veterans just lost their agency overnight and became puppets? Nah.
greatgib 5y ago
This is plain wrong. The code of systemd was already shit from the beginning. Random system breakages for stupid reasons. And everything hardcoded the ugliest way possible so that it is impossible to have alternate systems like some embedded ones.
- gotar 5y ago
  You've probably looked at wrong repo. Any example of ugly code?

goodpoint 5y ago

> enough to remember We are talking 10 years ago, not 1975 here. Unless everybody is 25 years old on HN...

int_19h 5y ago
Or they weren't using Linux 10 years ago?

jandrese 5y ago

SystemD is kind of like MacOS, if you have a typical use case then it makes everything very easy. If you are doing something weird it can make your life a living hell.

Denvercoder9 5y ago
> If you are doing something weird it can make your life a living hell. I find it much easier to reliably accomplish non-standard things with systemd unit definitions, than it used to be with the ill-defined complex set of shell scripts we had before.
- jcrawfordor 5y ago
  This issue is a lot bigger outside of services... I think the systemd service architecture is a clear improvement over SysV. The bigger issues I run into with systemd flexibility are related to the many other aspects of it. systemd-resolved, for example, has a very "opinionated" (to be polite) set of expectations about the environment and the systemd project tends to view any complaints about it not working outside of that environment as being the fault of the complainant. This leads to, well, DNSSEC is basically just broken and split horizon DNS is extremely unreliable. The former is sad but mostly doesn't matter, the latter leads to a lot of corporate networks having to disable resolved (I think configuring it to remove the semi-hardcoded "backup" resolvers reliably fixes this problem but honestly I find it completely ridiculous that resolved has a hardcoded list of DNS servers it just uses instead sometimes. It's unclear to me whether or not that's a bug at this point and I got tired of trying to follow the issues and mailing list threads where the developers were, uh, not amazingly helpful). systemd management of mounts can be similarly narrow about the types of configurations it supports, and systemd-firewalld just sort of openly only claims it can support simple use-cases. systemd-journald is also a net loss of flexibility compared to rsyslog but, on the other hand, rsyslog could quickly turn into an inscrutable mess if you used any of the advanced features (rainerscript...), so this may not be an entirely bad thing. And in general systemd is, well, opinionated. I hate to bang on resolved too much but I just happen to have spent quite a few hours last week figuring out resolved problems. Resolved does not handle "dotless" domains correctly in a lot of existing environments (it's very particular about exactly how the search domain is set up). Poettering has basically responded that it's because dotless domains are stupid and no one should use them, so it won't be changed. I don't necessarily disagree that dotless domains are not a good idea today but it does mean that resolved breaks a lot of older corporate and institutional environments that have been using them successfully for decades. This manifests as "I updated my distro and the intranet stopped working." That kind of breaking change is not very common with core Linux services and isn't going to make many friends in the IT crowd.
  - mianos 5y ago
    I think systemd is OK for a lot of things, I don't even hate journald. The API pretty good. But, I am 100% agreement on the whole resolvd thing. It is a complete fiasco for anything but someone's idea of a standard network. Even the most basic things, like waiting for DNS to be up before mounting a network filesystem has to be done by writing your own units if you don't want to just try and mount and hope for the best.
  - crazy_hombre 5y ago
    There's no such thing as systemd-firewalld.
    - jcrawfordor 5y ago
      Yeah, sorry, I'm blaming more on Poettering than is really fair. But firewalld is closely coupled with the overall systemd architecture and the projects are interconnected.
  - panick21_ 5y ago
    Systemd-resolved has some good ideas but can be a bit confusing. Overall its not bad, but it is a bit strange.
Gigachad 5y ago
Can you give some examples?
- jandrese 5y ago
  Trying to set up network booting with NFS mounted root with SystemD is a challenge to say the least.
- hedora 5y ago
  Not the person you’re responding to, but try getting it to mount home directories over NFS. I’m sure there are many other examples. Honestly, even after 10 years, I still have’t had a single positive interaction with it vs. traditional init. From what I’ve seen, old-school init handled (and continues to handle) the problems systemd “solved” more robustly and elegantly. I don’t get it.
  - bonzini 5y ago
    I'm not sure what's hard about mounting home directories over NFS?
gotar 5y ago
On the contrary - everything in systemd is much easier than before. Consider a situation, where your sh-backed SysV init was missing some function you required. Something between start-stop-daemon() or ifup(). You could throw a bunch of shell commands (note: most shell "programmers" are not capable of doing it properly; they do some '==' comparisons in /bin/sh shebang, don't handle errors, ignore fact that variables might contain "funny" characters etc. - but in general most of the scripts I've seen during my 24 years in Linux was simply badly written, meaning at least using bashizms). And then ...you watch out for every single package update that would overwrite your changes. Or you could polish them and try to push upstream. Now the situation seems to be worse - you can't just edit some random file to add missing function, you need to write it in source C files and recompile. Not a way for quick and dirty solution. But instead, you can simply write the same (s)hell commands in a script and call it from appropriate unit. No messing with thousand-lines scripts, no risk of overwrites, no need for upstreaming.

imoverclocked 5y ago

Network configuration (on say, Debian) is still a pain and has a lot of different ways of accomplishing similar tasks. Anything beyond the plain vanilla singular network device that is either dhcp/static is often painful. Systemd has some sauce, there is legacy /etc/network* stuff, there are daemons which can be configured graphically… it’s still a mess. In this arena, I think systemd has failed for the last decade. It introduced yet another way of doing things without considering the whole scope of what it needed to replace. I’m old enough to remember the culture wars and even older to remember a well thought out approach to overall configuration in Debian given the available tools of the time.

taftster 5y ago
Yes, this. The problem with systemd isn't systemd. It's that it never really replaced the other "legacy" parts effectively. Networking is a fantastic example. I'd be OK if there was one-definitive-place to configure networking, but when I read an article or follow a stack-overflow post, I'm not sure if what I'm changing is the most correct change or if its deprecated in favor of something newer. Systemd, I thought, would help settle this problem, but instead has only contributed to it.
- massysett 5y ago
  Isn’t the number one problem here that you’re following a Stack Overflow post? Your system should have good documentation on how to configure networking, or some obvious sanctioned way to do it, such as a GUI widget. If your system has neither, that’s the problem. If it has one or the other and you’re using Stack Overflow anyway, that’s the problem. Note I said “good” documentation - many Linux systems have lousy documentation and they lack an obvious, sanctioned way to do it. Again, that’s the problem.
jasonjayr 5y ago
Networking on linux is wildly powerful. Most desktop/SOHO installations would just be Wifi + DHCP. Maybe some VPN in the mix. But on the other side of the scale, there are routers with complex netfiltering, vlans, crazy GRE Tunnels, Tun/Tap, Wireguard, macvlans split between namespaces, bridges, etc, you name it. Defining a straightforward configuration structure for that complexity is going to be insane. NetworkManager + Debian's /etc/network/interfaces (IMHO) seems to be a good fit: auto config w/ a GUI in NM in the simple case, and if the interface is listed in /etc/network/interfaces, you can do it anyway you want with Debian's system, with pre-up, up, post-up, post-down, etc scripts to your heart's content to setup whatever complexity you need.

throwaway984393 5y ago

Part of it was political, but it was also just practical. Poettering's designs are some of the worst for usability I've ever seen. They technically work, but you need a million StackOverflow Q&As to figure out how to get anything done or solve the many common problems. It's like somebody decided that the problem with Linux was it wasn't complicated enough. And then everbody just went with it.

erik_seaberg 5y ago
I keep coming back to the https://news.ycombinator.com/item?id=19023885 https://news.ycombinator.com/item?id=19023885 issue. There’s a default setting somewhere that makes systemd knowingly kill normal user processes it did not start. Why does that exist? It was never init’s job, and init is not something end users should have to be aware of to run stuff with POSIX nohup.
- throwaway984393 5y ago
  Jesus. I didn't know about that, but it totally makes sense. Like when systemd devs told Linus he should change his kernel to fix a bug in systemd.

fartcannon 5y ago

I still don't trust it, given how aggressively it was pushed.

mihcsab 5y ago

My problem is that things start to depend on it, like snapd... I was looking into running anbox on wsl today, but I realized I need to compile stuff, because they distribute anbox in snap form only, so I kind of gave up...

mariusor 5y ago

> the way systemd developers pushed the community to adopt systemd, specially when they asked for 3rd party developers to make systemd a hard dependency. Do you have any links for these statements? My memory is not great but I think I would remember anything similar to this.

pas 5y ago

> systemd developers pushed the community [citation needed] I remember spending days reading the arguments on the Debian mailing list. There was no push from systemd devs. (Endless threads about upstart, how tweaking upstart would solve everything and it's already in Ubuntu, and a lots of questions about guarantees. Are openrc/upstart/systemd maintainers willing to guarantee this or that? And as far as I remember, there were no guarantees, obviously.) Before that when the hwdb and udev merged into systemd, people got the pitchforks out, it was the decision by Kay, but obviously "systemd devs" were somehow behind it. When Greg KH supported kdbus, somehow it was again those meddling systemd devs. Somehow OSS veterans just lost their agency overnight and became puppets? Nah.

greatgib 5y ago

This is plain wrong. The code of systemd was already shit from the beginning. Random system breakages for stupid reasons. And everything hardcoded the ugliest way possible so that it is impossible to have alternate systems like some embedded ones.

gotar 5y ago
You've probably looked at wrong repo. Any example of ugly code?