15 ms·

Broken by design: systemd

172 points by uggedal 13y ago

Tuna-Fish 13y ago
This reeks of the same kind of argument that was used to push for microkernels. Arbitrarily cutting up processes into smaller parts is not the same as decoupled design.
- d0 13y ago
  It's not arbitrary. There are logical security, reliability and convention boundaries that systemd doesn't consider which are raised in this article. Microkernels are a different subject altogether. To be honest, the only service management solution I've seen that I actually like for Unix platforms is BSD's simple rc and daemontools on top of it for process management.
- makomk 13y ago
  I don't think that's the point of the argument - after all, systemd is already arbitrarily cut up into smaller processes, it's just that they're all tightly coupled and nonfunctional on their own.
nailer 13y ago
The argument re: larger surface for updates is true. Systemd is smaller than the things it replaces, and consistent in itself, but all the logic is in PID 1. I fail to see a better alternative though. Old, disparate init systems repeat logic for: - starting services (the same init script copied 90 times, often badly) - journalling (unmaintained syslogd facilities means apps all report themselves as local0 and five different fixes for that) - a billion different 'sentinel / watch' type apps - various hacks on hacks for dependencies, parallel init, etc. If you rip out all the redundancy and replace them with something unified, that unified functionality can (and should) exist at low level. Maybe the argument is that a better way of handling systemd updates is needed?
- d0 13y ago
  These problems are all solved without destroying the Unix philosophy completely: starting services - daemontools. journaling - daemontools billion different sentinel/watch type apps - daemontools. dependencies - just don't do it. This is the old age of power sequencers again. It's coupling. Daemontools is a set of tiny programs that talk to each other, are fully privilege separated and guarantee reliability. Why they can't do this for systemd, I don't know. If you remove the DJB path conventions from it (the major objections), it's the right tool for the job.
  - tc104 13y ago
    Oh don't start babbling about Unix Philosophy. It's wearing a little thin. Even the people who actually worked on Unix are sick of hearing this appeal-to-authority, "BECAUSE UNIX PHILOSOPHY" nonsense by now.
  - FooBarWidget 13y ago
    > starting services - daemontools Daemontools is great, but please don't pretend it's a proper general-purpose init system. How do you start a certain service before another one? I suppose you can hack that inside the run script but then you're yet again duplicating logic, badly. > journaling - daemontools What? Daemontools provides a nice utility for logging a process stdout/stderr to a file. That's it. That doesn't come close to the problem that journald tries to solve. > billion different sentinel/watch type apps - daemontools. This is about the only thing that daemontools solves. > dependencies - just don't do it. This is the old age of power sequencers again. It's coupling. And here comes the mythical "coupling is bad" statement while completely ignoring the problem. No, I don't want my NFS daemon to be started before my network connection is up, thank you very much. Your post shows a complete lack of understanding of the problem.
    - d0 13y ago
      > Daemontools is great, but please don't pretend it's a proper general-purpose init system. How do you start a certain service before another one? I suppose you can hack that inside the run script but then you're yet again duplicating logic, badly. I'm not saying its a replacement. The model is suitable as a replacement i.e. inspiration should be taken from it. What? Daemontools provides a nice utility for logging a process stdout/stderr to a file. That's it. That doesn't come close to the problem that journald tries to solve. It guarantees that the logs are stamped externally, are committed to disk and that the logs haven't been tampered with by isolating them from other processes and user accounts that can modify or write to them. Don't need journaling on top, just separation of concerns. Indexing - don't need it. No one does. That's an external problem solved by syslog collectors like Splunk. As someone who deals with up to 500Gb of logs a day, I know my shit. Systemd doesn't solve a thing here. In fact it adds overhead to a solved problem. And here comes the mythical "coupling is bad" statement while completely ignoring the problem. No, I don't want my NFS daemon to be started before my network connection is up, thank you very much. Well what the hell does your NFS daemon do when your network goes down, your adapter gets hotplugged after a failure or someone falls over the cable? There is so little of the problem solved by systemd it's unbelievable. The RIGHT solution is to make the processes resilient to failure conditions like this.
scrollaway 13y ago
Discussion on reddit: https://pay.reddit.com/r/programming/comments/1xggg8/broken_by_design_systemd/ https://pay.reddit.com/r/programming/comments/1xggg8/broken_... On that note, nothing screams FUD like a screenshot of Windows behaviour on an article like that. Is there a Linux-Windows equivalent to Godwin's law?
- baq 13y ago
  the picture is very adequate when paired with this sentece: > Unfortunately, by moving large amounts of functionality that's likely to need to be upgraded into PID 1, systemd makes it impossible to upgrade without rebooting. This leads to "Linux" becoming the laughing stock of Windows fans, as happened with Ubuntu a long time ago.
bitwize 13y ago
Nice article, but at this point it's game, set, match -- systemd has won. The Linux community overwhelmingly supports it, and its advantages clearly outweigh its perceived disadvantages. "It's not Unix" is no longer a valid critique. The old Unix design philosophy is dead. People want systems of components that function as an integrated whole. They do NOT want pieces that are generally unaware of each other (this is touted as the virtue of "loose coupling"). Yes, pid 1 does a lot. That's by necessity. It's not your grandpap's Unix anymore. Do you want to control all the system services, including the "badly behaved" ones? That functionality needs to be in pid 1. Do you want to bring the system up safely and sanely, accounting for things like hotplugged disks in /etc/fstab? That functionality needs to be in pid 1. Systemd is the correct design for modern Linux. It also brings the advantage of better integration and greater commonality across distros.
- icebraining 13y ago
  Why do those things need to be on pid 1? Just asserting it is not really convincing.
  - sandGorgon 13y ago
    Any init system needs to be pid 1 [1] This blog post does not identify the subtle argument that upstart maintainers made against systemd at pid -1 : it also brings along with all the other services (like logind) that frameworks like Gnome depend on. So, if Gnome wants to depend on logind and logind depends on systemd (post systemd rev-205), this means that Gnome is irrevocably coupled to systemd which must run as PID-1 [2]. This means it becomes hard for upstart to be used with Gnome. There has been no doubt whatsover that an init system needs to run as PID-1. [1] http://en.wikipedia.org/wiki/Process_identifier#Unix-like http://en.wikipedia.org/wiki/Process_identifier#Unix-like [2] I must add here that Gnome does include a fallback path to not depend on systemd components.
    - vidarh 13y ago
      A part of any init system needs to be pid 1. The question is how much of it, and whether or not Systemd piles too much stuff into pid 1 that could instead be farmed out to separate processes, handled by separate binaries, that could be more easily upgraded or replaced. (I don't know; I've not looked at Systemd enough to have an opinion)
ANTSANTS 13y ago
This was posted earlier today, under the exact same title, but with a URL that differed solely by a trailing backslash. It hit the front page, too! Way to go, HN duplicate detector. https://news.ycombinator.com/item?id=7207655 https://news.ycombinator.com/item?id=7207655
oofabz 13y ago
> PID 1 brings down the whole system when it crashes. This matters because systemd is complex. The Linux kernel is much more complex and also brings down the whole system when it crashes. > Attack Surface Sysvinit runs shell scripts and has known race conditions. A system like that is not secure by design. It is as secure as it is because exploits were fixed one by one over the years. > Reboot to Upgrade You must also reboot to upgrade sysvinit.
- InclinedPlane 13y ago
  See what happens when you kill PID 1 on a system using sysvinit.
  - munchhausen 13y ago
    On Linux, nothing: The only signals that can be sent to process ID 1, the init process, are those for which init has explicitly installed signal handlers. This is done to assure the system is not brought down accidentally. (From kill(2)) In practice, you will find that: kill(1, SIGKILL) = 0 results in exactly...no effect.
    - nailer 13y ago
      It's also because Unix doesn't generally believe in being forgiving (or, UX in general): someapp & [job 1 backgrounded] kill %1 (job 1 is killed - yep, you can use kill for jobs) Miss a percent there? You've just sent a TERM to init.
      - srd 13y ago
        Which won't matter if you're a normal user. If you're root - why are you using the job system to kill your jobs?
        nailer 13y ago
        > If you're root - why are you using the job system to kill your jobs? Probably because you know the features of your shell and enjoy productivity.
dschiptsov 13y ago
It is broken by design because it contradicts with some general principles, such as "single responsibility", "share nothing", which leads to "divide et impera" and other "nice things". Unix philosophy, according to which a system is a set of lose-coupled tools, each of which, at least in theory, is supposed to do just one thing, but do it well, and to follow certain conventions, such as using text streams as input and output, which allows them to be hooked together into pipelines, is a really great one. To sacrifice it just for very contradictional arguments of people, who, perhaps, not fully understand it or its consequences for a complex system's design, is, at least, shortsighted decision. (look what a wonder is Plan9, at least from the system design perspective and "less is more" principle). As for so-called "problems", BSD people would tell you that most of these "problems" are mere creations of distracted minds.) Introducing various daemons and ctls merely adds unnecessary complexity compared with the uniformity of "just shell scripts" and a very simple set of conventions. At least everything works fine with FreeBSD and its huge ports collection.
- JulianMorrison 13y ago
  We're dealing with irreducible complexity here. Unix philosophy keeps the complexity out of the components - fair enough. But that merely means that the complexity reappears in the glue code (sysvinit and friends). And glue code is harder to secure, being scattered, disparate, and executing in complicated VMs such as the shell-plus-kernel, and therefore hard to reason about.
  - dschiptsov 13y ago
    FreeBSD, OpenBSD, Plan9, AIX (Solaris decision to use Java-based tool was plainly idiotic) for some reasons would think otherwise.
    - gtirloni 13y ago
      Which Java-based tools is Solaris using in this area? I don't think SMF is Java-based. SMF is pleasant to use and Linux should have copied it entirely, including syntax, but they didn't (NIH syndrome?). http://en.wikipedia.org/wiki/Service_Management_Facility http://en.wikipedia.org/wiki/Service_Management_Facility systemd is really a pain to use. I'm still amazed at how quick everybody was at adopting it. Perhaps people were really desperate to ditch the sysvinit stuff.
olgeni 13y ago
Why not copy SMF?
- nailer 13y ago
  systemd shares much of its design with launchd and SMF, with the obvious exception of using XML for service config. The same arguments against it would apply.
chipsy 13y ago
I get it now. The distro ecosystem has been moving towards the other half of Sutherland's wheel of reincarnation - in this case, from modular systems towards monolithic ones. First it was stuff in the desktop space - now it's creeping into other areas. You can state the particular point of inflection as a terrible thing, or a good thing, but seen as a technology learning cycle, it becomes _necessary_ even if the implementation chosen is shit and you hate everything about it. At the other end of it, it still gets broken up again into a different set of modular components whose design wouldn't have been apparent at the beginning. Since the initial discussion stage is over, I would say, if you really think this is the wrong solution(and I don't consider myself expert enough on inits to cast much judgment here), your work is best spent on the code environment immediately surrounding systemd - not direct competitors, but stuff that it has to deal with at some point. That will put pressure on the design to either break entirely or evolve to support whatever you've come up with - and when that happens you get to walk in with a patch to show and kick off another firestorm.
CopperWing 13y ago
Lot's of FUD there on ewontfix.com. A better source of information: https://wiki.debian.org/Debate/initsystem/systemd https://wiki.debian.org/Debate/initsystem/systemd
Aardwolf 13y ago
> The desktop is quickly becoming irrelevant. The future platform is going to be mobile and is going to be dealing with the reality of running untrusted applications This is off topic (tho from the article :p), but, is it really becoming irrelevant? What will replace such things as: typing lots of text, photoshopping/gimping, programming IDE, browsing with multiple browser windows with dozens of tabs each, etc...? Would Linux users abandon the desktop, if so, for what? P.S. unless I can plugin my phone into something with a keyboard, mouse and big monitors (or more productive equivalent), it is as fast as my current desktop, and has desktop oriented software in that mode, I don't see myself abandoning the desktop
- perlgeek 13y ago
  The desktop isn't hip, and it's not a source of growing amount of sells; but most people I know still spend at least 6 hours a day in front of a desktop computer, with no indication that it'll change. So my own answer is a firm "no".
- tinco 13y ago
  Desktops are not really going away, they're just taking a roundabout. Instead of desktops people are now buying laptops, and attaching external monitors, keyboards, mice. The next step could be people buying tablets, and attaching the external devices for power use. The difference is subtle, as it seems to be nothing is materially changing, but the current tablet/mobile ecosystem is dominated by application container operating systems like Android and iOS, and it seems unlikely that that is going to change. This is the movement that is obsoleting traditional Linux and Windows, at least on consumer hardware. So to answer your question: "typing lots of text, photoshopping/gimping, programming IDE, browsing with multiple browser windows" will be replaced by functionality in containerized applications or 'apps'. The apps are bought/downloaded from app repositories that are loosely maintained by commercial entities, and they could very well be untrustworthy. As to your P.S.: yes, exactly, so you already know you are going to abandoning the desktop.
  - Aardwolf 13y ago
    > As to your P.S.: yes, exactly, so you already know you are going to abandoning the desktop. Oh, hmm, would I guess also really depend on what OS and software is available for it and how open the whole ecosystem is, I mean, something unix-like that you can modify, not something single-closed-source-app-at-a-time-like :) And if a desktop machine will allow you to stuff many more powerful CPUs and GPUs in it I may keep using one anyway :D
chroma 13y ago
My biggest concern with systemd has to do with its creator, Lennart Poettering. He created PulseAudio and Avahi. Neither of those are particularly reliable, stable pieces of software. Also, Lennart's opposition to making systemd portable is worrying. The other unixes have it hard enough already. I don't want to single him out and hate on him. Lennart's probably a nice guy, and he's contributed a lot more to open source than I have. But having the people responsible for PulseAudio involved in the next-generation init sounds like a recipe for disaster.
- VLM 13y ago
  A minor disagreement, that software is perfectly reliable with decent hardware and perfect 3rd party software, it just does not degrade gracefully (to say the least) when things it interoperates with misbehave. If you're not doing networking or audio, it doesn't matter if buggy hardware makes PA go bonkers or weird service defs make avahi behave weirdly. A lack of error handling and error recovery. Its a violation of the old unix principle of be liberal in what you accept. The most likely outcome of that philosophy installed in everyone's init system is going to be a heck of a lot of systemd crashes, although it'll all technically have a root cause of some other software or perhaps dodgy hardware. None of those crashes would occur with an init system that handles errors better, so systemd is going to catch a lot of blame.
  - makomk 13y ago
    Actually, the flat volumes feature that's on by default in recent PulseAudio versions (and cannot be disabled without editing a config file as root) relies on entirely unspecified behaviour of the audio hardware, namely the timing relationship between volume changes and sample data. It has a hack that avoids deafening people unexpectedly on most hardware, but at the expense of very audible volume glitches every time it decides to fiddle with the global volume. (Also, thanks to inconvenient analog issues like component tolerances, the most reliable way for hardware to deal with PulseAudio's flat volumes is to provide no volume controls except for a single digital one that simply undoes all of PulseAudio's weird fiddling. This volume control would provide no functionality that couldn't just be implemented in software, but PulseAudio handles devices without one really badly.)
VLM 13y ago
The scariest part no one is talking about is the feature creep. All software eventually dies; systemd will be no different. Some folks hope that happens immediately, some hope we're "stuck" with it for a couple generations, maybe 30 years. Regardless of exactly when, its going to be replaced someday, and massive feature creep seems intentionally designed to make the eventual changeover as painful as possible. Someday in the future, either 2016 or 2066 someone is going to want to rearchitect the "thing that the kernel starts first" and is going to freak out at the insane feature creep... so you're telling me that I not only have to create something to start services which is no big deal, but also an entire logging system, and a hotplug handler, and a cgroup server, and ... WTF does this blasted thing have an MP3 player, a web server, and an IRC client in it too? Its an awful design but probably survivable, at least temporarily, and once the flamewars die down a sensible replacement with a good design can be rolled out to replace it. But mid-flamewar with all manner of business / marketing money flying everywhere on all sides is not the ideal blue sky redevelopment time. As much as it pains me to say it, rolling out an init that has almost the worst possible inherent design possible does have some value to the ecosystem as a counterexample, here's what not to do with 2.0. Obviously its replacement will be better, I just hope it doesn't swing too far in the opposite direction as a reaction. The point of this paragraph is the most important thing to do at this time is to lay the seeds for the replacement, while not over-reacting to the disaster of systemd. So, the replacement for systemd, to be rolled out sometime between 2015 and 2065, should ... (insert feature list here).
- marcosdumay 13y ago
  Great, now what will think the unluky sould that'll need to replace Linux? The problem we are seeing is the framework vs. libraries dicotomy. Cut the features in too many packages, and you won't be able to use them anymore. Integrate them too much, and you won't be able to maintain them anymore. Anyway, just like the kernel, some day we must be able to say "launching the basic processes is a solved problem", and actualy solve it in a way that does not need to be replaced. We are not there, but we are in a point where the feature set is quite stable, so I guess we can improve usability a bit, at the expense of maintanability.
gegenschall 13y ago
I guess this post has some relevance in this thread: http://0pointer.de/blog/projects/the-biggest-myths.html http://0pointer.de/blog/projects/the-biggest-myths.html
- pekk 13y ago
  Some of those jokes are very funny. I really liked the one about how systemd is modular, just in the same way that the Linux kernel is modular, because you can compile it with all different options.
  - gegenschall 13y ago
    I do not really understand what you're saying. 1) This is actually modular. I mean... those things are even called "modules". 2) Even if it wasn't modular, how would it matter? You're comparing to the Linux kernel. If that one isn't modular then according to you, systemd also isn't modular. Allright, so be it, did you have any problems with missing modularity in the Linux kernel lately?
Sammi 13y ago
This feels like the Tanenbaum–Torvalds debate all over again.
julie1 13y ago
The major problem is the requirements and the system dependencies it introduces on a core component. http://harmful.cat-v.org/software/dynamic-linking/ http://harmful.cat-v.org/software/dynamic-linking/ systemd introduces a major PITA for the OS packager by putting a lot of dependency on a lot of parts with "moving components" making it both more vulnerable as 1 and more likely to fail (it is not complex it is complicated). Systemd also tends to become a system requirements for a lot of packages making it unsuppressable. It makes the problem of dependency solving logically totally messy without justifying this cost by any real improvement. It is introducing the DLL hell for the most crucial part of the system (PID 1).
peterwwillis 13y ago
I won't ever use a distribution that uses systemd as pid 1. There is only one reason for this. I want to be in control how my system works. To do that I need to be sure of what it is doing and how. As the complexity of a system grows, the number of things it does under the hood grows. At some point I can't keep track of what's going on, and I no longer have confidence in my knowledge of the system, without setting aside time for a code review. With pid 1 being incredibly simple, and the executables it runs being incredibly simple, I know what's going on, I know what the possible problems are and I know how to fix them. The more complex it is, the more difficult it is for me to know what's going on. That's all it comes down to for me. As an aside: A lot of systemd is API-driven, meaning if you want a feature you have to build it into the system as new code. This means you have to be a C developer to leverage some new feature, instead of writing an interpreted script and interacting with it via pipes/plugins/etc. If your current system is dependent on some feature that doesn't yet exist in systemd, you now have to build it in, instead of using what worked with sysvinit before. Not only is systemd broken by design, it's still in its infancy, and does not support the same features that have been built into other systems for years. This additional load of development for many orgs serves no purpose, and is essentially ignored by the leadership that decide to change init systems by fiat.
- hdevalence 13y ago
  > I want to be in control how my system works. To do that I need to be sure of what it is doing and how. As the complexity of a system grows, the number of things it does under the hood grows. At some point I can't keep track of what's going on, and I no longer have confidence in my knowledge of the system, without setting aside time for a code review. > With pid 1 being incredibly simple, and the executables it runs being incredibly simple, I know what's going on, I know what the possible problems are and I know how to fix them. The more complex it is, the more difficult it is for me to know what's going on. That's all it comes down to for me. How is, say, sysv init better in this regard? Instead of having declarative service files that say what should be done, you have a collection of disparate shell scripts, all with their own implementations of basically the same things. The complexity isn't reduced, it's spread around (and IMO it's more complicated). If you want to have control over your system, wouldn't you want to use an init system that can actually keep track of all of the child processes of a service? Brittle hacks like pidfiles cannot ensure management over all child processes of a service -- this is only possible with cgroups. > As an aside: A lot of systemd is API-driven, meaning if you want a feature you have to build it into the system as new code. This means you have to be a C developer to leverage some new feature, instead of writing an interpreted script and interacting with it via pipes/plugins/etc. If your current system is dependent on some feature that doesn't yet exist in systemd, you now have to build it in, instead of using what worked with sysvinit before. No, you just use D-Bus, from whatever your favorite language is, using that language's D-Bus bindings. Systemd has a stable D-Bus API for this purpose that's guaranteed to work in the future. > Not only is systemd broken by design, it's still in its infancy, and does not support the same features that have been built into other systems for years. Out of curiosity, what features are you referring to specifically? > This additional load of development for many orgs serves no purpose, and is essentially ignored by the leadership that decide to change init systems by fiat. This would be more convincing if it were just Lennart & co. making the decisions, but in the absence of evidence to the contrary, I'm going to go out on a limb and assume that the technical leadership of Fedora, RHEL, OpenSUSE, Arch, and now Debian do not all make decisions by fiat for no reason.
mwcampbell 13y ago
The main systemd binary (version 208 for x86_64 as shipped by Arch Linux) is 1020 KB. I get that the systemd package is really several binaries, but still, does the binary for PID 1 need to have that much in it?
- antientropic 13y ago
  That's about 1/8000th of the RAM in my laptop. Is 1020 KB really such a big deal?
  - mwcampbell 13y ago
    The problem is that software complexity grows exponentially with code size. Well, maybe not exponentially, but more than linearly. And the article established that it's important to keep PID 1 simple.
arca_vorago 13y ago
Stuff like this is why I am increasingly getting back into BSD and recently Minix3. I think this is just part of a larger issue with the *nix ecosystem, where lots of philosophical and design assertions are proving to no longer work, while some are falsely being said to no longer work but still do. With the linux kernel now at over 10 million lines of code, the idea of peer review of source code seems too big of a job for random unpayed volunteers. I think we need to move back to simplicity and decentralization as much as possible, because the problem with centralization is that it makes for a weak system.
brokenparser 13y ago
Systemd makes my life difficult. Case in point: I want to stop cups, so I run "sudo service cups stop". This appears to work, but then it prints out a warning along the lines of: cups was stopped, but it can still be activated by: - cups.sprocket - cups.latch What are these things and why are they there at all? How do I shut them off? It doesn't tell, so stopping a service has become pointless. If I tell it to stop, make it stop!
njharman 13y ago
systemd reminds me of 90's era Microsoft's "release a new service/api/whatever that is almost the same, but different enough to create incompatibility and lockin, include small addition or improvement to encourage adoption, make everything require it to force adoption by dissenters." I'm not joking.

172 points by uggedal 13y ago

This reeks of the same kind of argument that was used to push for microkernels. Arbitrarily cutting up processes into smaller parts is not the same as decoupled design.

d0 13y ago
It's not arbitrary. There are logical security, reliability and convention boundaries that systemd doesn't consider which are raised in this article. Microkernels are a different subject altogether. To be honest, the only service management solution I've seen that I actually like for Unix platforms is BSD's simple rc and daemontools on top of it for process management.
makomk 13y ago
I don't think that's the point of the argument - after all, systemd is already arbitrarily cut up into smaller processes, it's just that they're all tightly coupled and nonfunctional on their own.

nailer 13y ago

The argument re: larger surface for updates is true. Systemd is smaller than the things it replaces, and consistent in itself, but all the logic is in PID 1. I fail to see a better alternative though. Old, disparate init systems repeat logic for: - starting services (the same init script copied 90 times, often badly) - journalling (unmaintained syslogd facilities means apps all report themselves as local0 and five different fixes for that) - a billion different 'sentinel / watch' type apps - various hacks on hacks for dependencies, parallel init, etc. If you rip out all the redundancy and replace them with something unified, that unified functionality can (and should) exist at low level. Maybe the argument is that a better way of handling systemd updates is needed?

d0 13y ago
These problems are all solved without destroying the Unix philosophy completely: starting services - daemontools. journaling - daemontools billion different sentinel/watch type apps - daemontools. dependencies - just don't do it. This is the old age of power sequencers again. It's coupling. Daemontools is a set of tiny programs that talk to each other, are fully privilege separated and guarantee reliability. Why they can't do this for systemd, I don't know. If you remove the DJB path conventions from it (the major objections), it's the right tool for the job.
- tc104 13y ago
  Oh don't start babbling about Unix Philosophy. It's wearing a little thin. Even the people who actually worked on Unix are sick of hearing this appeal-to-authority, "BECAUSE UNIX PHILOSOPHY" nonsense by now.
- FooBarWidget 13y ago
  > starting services - daemontools Daemontools is great, but please don't pretend it's a proper general-purpose init system. How do you start a certain service before another one? I suppose you can hack that inside the run script but then you're yet again duplicating logic, badly. > journaling - daemontools What? Daemontools provides a nice utility for logging a process stdout/stderr to a file. That's it. That doesn't come close to the problem that journald tries to solve. > billion different sentinel/watch type apps - daemontools. This is about the only thing that daemontools solves. > dependencies - just don't do it. This is the old age of power sequencers again. It's coupling. And here comes the mythical "coupling is bad" statement while completely ignoring the problem. No, I don't want my NFS daemon to be started before my network connection is up, thank you very much. Your post shows a complete lack of understanding of the problem.
  - d0 13y ago
    > Daemontools is great, but please don't pretend it's a proper general-purpose init system. How do you start a certain service before another one? I suppose you can hack that inside the run script but then you're yet again duplicating logic, badly. I'm not saying its a replacement. The model is suitable as a replacement i.e. inspiration should be taken from it. What? Daemontools provides a nice utility for logging a process stdout/stderr to a file. That's it. That doesn't come close to the problem that journald tries to solve. It guarantees that the logs are stamped externally, are committed to disk and that the logs haven't been tampered with by isolating them from other processes and user accounts that can modify or write to them. Don't need journaling on top, just separation of concerns. Indexing - don't need it. No one does. That's an external problem solved by syslog collectors like Splunk. As someone who deals with up to 500Gb of logs a day, I know my shit. Systemd doesn't solve a thing here. In fact it adds overhead to a solved problem. And here comes the mythical "coupling is bad" statement while completely ignoring the problem. No, I don't want my NFS daemon to be started before my network connection is up, thank you very much. Well what the hell does your NFS daemon do when your network goes down, your adapter gets hotplugged after a failure or someone falls over the cable? There is so little of the problem solved by systemd it's unbelievable. The RIGHT solution is to make the processes resilient to failure conditions like this.

scrollaway 13y ago

Discussion on reddit: https://pay.reddit.com/r/programming/comments/1xggg8/broken_by_design_systemd/ https://pay.reddit.com/r/programming/comments/1xggg8/broken_... On that note, nothing screams FUD like a screenshot of Windows behaviour on an article like that. Is there a Linux-Windows equivalent to Godwin's law?

baq 13y ago
the picture is very adequate when paired with this sentece: > Unfortunately, by moving large amounts of functionality that's likely to need to be upgraded into PID 1, systemd makes it impossible to upgrade without rebooting. This leads to "Linux" becoming the laughing stock of Windows fans, as happened with Ubuntu a long time ago.

bitwize 13y ago

Nice article, but at this point it's game, set, match -- systemd has won. The Linux community overwhelmingly supports it, and its advantages clearly outweigh its perceived disadvantages. "It's not Unix" is no longer a valid critique. The old Unix design philosophy is dead. People want systems of components that function as an integrated whole. They do NOT want pieces that are generally unaware of each other (this is touted as the virtue of "loose coupling"). Yes, pid 1 does a lot. That's by necessity. It's not your grandpap's Unix anymore. Do you want to control all the system services, including the "badly behaved" ones? That functionality needs to be in pid 1. Do you want to bring the system up safely and sanely, accounting for things like hotplugged disks in /etc/fstab? That functionality needs to be in pid 1. Systemd is the correct design for modern Linux. It also brings the advantage of better integration and greater commonality across distros.

icebraining 13y ago
Why do those things need to be on pid 1? Just asserting it is not really convincing.
- sandGorgon 13y ago
  Any init system needs to be pid 1 [1] This blog post does not identify the subtle argument that upstart maintainers made against systemd at pid -1 : it also brings along with all the other services (like logind) that frameworks like Gnome depend on. So, if Gnome wants to depend on logind and logind depends on systemd (post systemd rev-205), this means that Gnome is irrevocably coupled to systemd which must run as PID-1 [2]. This means it becomes hard for upstart to be used with Gnome. There has been no doubt whatsover that an init system needs to run as PID-1. [1] http://en.wikipedia.org/wiki/Process_identifier#Unix-like http://en.wikipedia.org/wiki/Process_identifier#Unix-like [2] I must add here that Gnome does include a fallback path to not depend on systemd components.
  - vidarh 13y ago
    A part of any init system needs to be pid 1. The question is how much of it, and whether or not Systemd piles too much stuff into pid 1 that could instead be farmed out to separate processes, handled by separate binaries, that could be more easily upgraded or replaced. (I don't know; I've not looked at Systemd enough to have an opinion)

ANTSANTS 13y ago

This was posted earlier today, under the exact same title, but with a URL that differed solely by a trailing backslash. It hit the front page, too! Way to go, HN duplicate detector. https://news.ycombinator.com/item?id=7207655 https://news.ycombinator.com/item?id=7207655

oofabz 13y ago

> PID 1 brings down the whole system when it crashes. This matters because systemd is complex. The Linux kernel is much more complex and also brings down the whole system when it crashes. > Attack Surface Sysvinit runs shell scripts and has known race conditions. A system like that is not secure by design. It is as secure as it is because exploits were fixed one by one over the years. > Reboot to Upgrade You must also reboot to upgrade sysvinit.

InclinedPlane 13y ago
See what happens when you kill PID 1 on a system using sysvinit.
- munchhausen 13y ago
  On Linux, nothing: The only signals that can be sent to process ID 1, the init process, are those for which init has explicitly installed signal handlers. This is done to assure the system is not brought down accidentally. (From kill(2)) In practice, you will find that: kill(1, SIGKILL) = 0 results in exactly...no effect.
  - nailer 13y ago
    It's also because Unix doesn't generally believe in being forgiving (or, UX in general): someapp & [job 1 backgrounded] kill %1 (job 1 is killed - yep, you can use kill for jobs) Miss a percent there? You've just sent a TERM to init.
    - srd 13y ago
      Which won't matter if you're a normal user. If you're root - why are you using the job system to kill your jobs?
      - nailer 13y ago
        > If you're root - why are you using the job system to kill your jobs? Probably because you know the features of your shell and enjoy productivity.

dschiptsov 13y ago

It is broken by design because it contradicts with some general principles, such as "single responsibility", "share nothing", which leads to "divide et impera" and other "nice things". Unix philosophy, according to which a system is a set of lose-coupled tools, each of which, at least in theory, is supposed to do just one thing, but do it well, and to follow certain conventions, such as using text streams as input and output, which allows them to be hooked together into pipelines, is a really great one. To sacrifice it just for very contradictional arguments of people, who, perhaps, not fully understand it or its consequences for a complex system's design, is, at least, shortsighted decision. (look what a wonder is Plan9, at least from the system design perspective and "less is more" principle). As for so-called "problems", BSD people would tell you that most of these "problems" are mere creations of distracted minds.) Introducing various daemons and ctls merely adds unnecessary complexity compared with the uniformity of "just shell scripts" and a very simple set of conventions. At least everything works fine with FreeBSD and its huge ports collection.

JulianMorrison 13y ago
We're dealing with irreducible complexity here. Unix philosophy keeps the complexity out of the components - fair enough. But that merely means that the complexity reappears in the glue code (sysvinit and friends). And glue code is harder to secure, being scattered, disparate, and executing in complicated VMs such as the shell-plus-kernel, and therefore hard to reason about.
- dschiptsov 13y ago
  FreeBSD, OpenBSD, Plan9, AIX (Solaris decision to use Java-based tool was plainly idiotic) for some reasons would think otherwise.
  - gtirloni 13y ago
    Which Java-based tools is Solaris using in this area? I don't think SMF is Java-based. SMF is pleasant to use and Linux should have copied it entirely, including syntax, but they didn't (NIH syndrome?). http://en.wikipedia.org/wiki/Service_Management_Facility http://en.wikipedia.org/wiki/Service_Management_Facility systemd is really a pain to use. I'm still amazed at how quick everybody was at adopting it. Perhaps people were really desperate to ditch the sysvinit stuff.

olgeni 13y ago

Why not copy SMF?

nailer 13y ago
systemd shares much of its design with launchd and SMF, with the obvious exception of using XML for service config. The same arguments against it would apply.

chipsy 13y ago

I get it now. The distro ecosystem has been moving towards the other half of Sutherland's wheel of reincarnation - in this case, from modular systems towards monolithic ones. First it was stuff in the desktop space - now it's creeping into other areas. You can state the particular point of inflection as a terrible thing, or a good thing, but seen as a technology learning cycle, it becomes _necessary_ even if the implementation chosen is shit and you hate everything about it. At the other end of it, it still gets broken up again into a different set of modular components whose design wouldn't have been apparent at the beginning. Since the initial discussion stage is over, I would say, if you really think this is the wrong solution(and I don't consider myself expert enough on inits to cast much judgment here), your work is best spent on the code environment immediately surrounding systemd - not direct competitors, but stuff that it has to deal with at some point. That will put pressure on the design to either break entirely or evolve to support whatever you've come up with - and when that happens you get to walk in with a patch to show and kick off another firestorm.

CopperWing 13y ago

Lot's of FUD there on ewontfix.com. A better source of information: https://wiki.debian.org/Debate/initsystem/systemd https://wiki.debian.org/Debate/initsystem/systemd

Aardwolf 13y ago

> The desktop is quickly becoming irrelevant. The future platform is going to be mobile and is going to be dealing with the reality of running untrusted applications This is off topic (tho from the article :p), but, is it really becoming irrelevant? What will replace such things as: typing lots of text, photoshopping/gimping, programming IDE, browsing with multiple browser windows with dozens of tabs each, etc...? Would Linux users abandon the desktop, if so, for what? P.S. unless I can plugin my phone into something with a keyboard, mouse and big monitors (or more productive equivalent), it is as fast as my current desktop, and has desktop oriented software in that mode, I don't see myself abandoning the desktop

perlgeek 13y ago
The desktop isn't hip, and it's not a source of growing amount of sells; but most people I know still spend at least 6 hours a day in front of a desktop computer, with no indication that it'll change. So my own answer is a firm "no".
tinco 13y ago
Desktops are not really going away, they're just taking a roundabout. Instead of desktops people are now buying laptops, and attaching external monitors, keyboards, mice. The next step could be people buying tablets, and attaching the external devices for power use. The difference is subtle, as it seems to be nothing is materially changing, but the current tablet/mobile ecosystem is dominated by application container operating systems like Android and iOS, and it seems unlikely that that is going to change. This is the movement that is obsoleting traditional Linux and Windows, at least on consumer hardware. So to answer your question: "typing lots of text, photoshopping/gimping, programming IDE, browsing with multiple browser windows" will be replaced by functionality in containerized applications or 'apps'. The apps are bought/downloaded from app repositories that are loosely maintained by commercial entities, and they could very well be untrustworthy. As to your P.S.: yes, exactly, so you already know you are going to abandoning the desktop.
- Aardwolf 13y ago
  > As to your P.S.: yes, exactly, so you already know you are going to abandoning the desktop. Oh, hmm, would I guess also really depend on what OS and software is available for it and how open the whole ecosystem is, I mean, something unix-like that you can modify, not something single-closed-source-app-at-a-time-like :) And if a desktop machine will allow you to stuff many more powerful CPUs and GPUs in it I may keep using one anyway :D

chroma 13y ago

My biggest concern with systemd has to do with its creator, Lennart Poettering. He created PulseAudio and Avahi. Neither of those are particularly reliable, stable pieces of software. Also, Lennart's opposition to making systemd portable is worrying. The other unixes have it hard enough already. I don't want to single him out and hate on him. Lennart's probably a nice guy, and he's contributed a lot more to open source than I have. But having the people responsible for PulseAudio involved in the next-generation init sounds like a recipe for disaster.

VLM 13y ago
A minor disagreement, that software is perfectly reliable with decent hardware and perfect 3rd party software, it just does not degrade gracefully (to say the least) when things it interoperates with misbehave. If you're not doing networking or audio, it doesn't matter if buggy hardware makes PA go bonkers or weird service defs make avahi behave weirdly. A lack of error handling and error recovery. Its a violation of the old unix principle of be liberal in what you accept. The most likely outcome of that philosophy installed in everyone's init system is going to be a heck of a lot of systemd crashes, although it'll all technically have a root cause of some other software or perhaps dodgy hardware. None of those crashes would occur with an init system that handles errors better, so systemd is going to catch a lot of blame.
- makomk 13y ago
  Actually, the flat volumes feature that's on by default in recent PulseAudio versions (and cannot be disabled without editing a config file as root) relies on entirely unspecified behaviour of the audio hardware, namely the timing relationship between volume changes and sample data. It has a hack that avoids deafening people unexpectedly on most hardware, but at the expense of very audible volume glitches every time it decides to fiddle with the global volume. (Also, thanks to inconvenient analog issues like component tolerances, the most reliable way for hardware to deal with PulseAudio's flat volumes is to provide no volume controls except for a single digital one that simply undoes all of PulseAudio's weird fiddling. This volume control would provide no functionality that couldn't just be implemented in software, but PulseAudio handles devices without one really badly.)

VLM 13y ago

The scariest part no one is talking about is the feature creep. All software eventually dies; systemd will be no different. Some folks hope that happens immediately, some hope we're "stuck" with it for a couple generations, maybe 30 years. Regardless of exactly when, its going to be replaced someday, and massive feature creep seems intentionally designed to make the eventual changeover as painful as possible. Someday in the future, either 2016 or 2066 someone is going to want to rearchitect the "thing that the kernel starts first" and is going to freak out at the insane feature creep... so you're telling me that I not only have to create something to start services which is no big deal, but also an entire logging system, and a hotplug handler, and a cgroup server, and ... WTF does this blasted thing have an MP3 player, a web server, and an IRC client in it too? Its an awful design but probably survivable, at least temporarily, and once the flamewars die down a sensible replacement with a good design can be rolled out to replace it. But mid-flamewar with all manner of business / marketing money flying everywhere on all sides is not the ideal blue sky redevelopment time. As much as it pains me to say it, rolling out an init that has almost the worst possible inherent design possible does have some value to the ecosystem as a counterexample, here's what not to do with 2.0. Obviously its replacement will be better, I just hope it doesn't swing too far in the opposite direction as a reaction. The point of this paragraph is the most important thing to do at this time is to lay the seeds for the replacement, while not over-reacting to the disaster of systemd. So, the replacement for systemd, to be rolled out sometime between 2015 and 2065, should ... (insert feature list here).

marcosdumay 13y ago
Great, now what will think the unluky sould that'll need to replace Linux? The problem we are seeing is the framework vs. libraries dicotomy. Cut the features in too many packages, and you won't be able to use them anymore. Integrate them too much, and you won't be able to maintain them anymore. Anyway, just like the kernel, some day we must be able to say "launching the basic processes is a solved problem", and actualy solve it in a way that does not need to be replaced. We are not there, but we are in a point where the feature set is quite stable, so I guess we can improve usability a bit, at the expense of maintanability.

gegenschall 13y ago

I guess this post has some relevance in this thread: http://0pointer.de/blog/projects/the-biggest-myths.html http://0pointer.de/blog/projects/the-biggest-myths.html

pekk 13y ago
Some of those jokes are very funny. I really liked the one about how systemd is modular, just in the same way that the Linux kernel is modular, because you can compile it with all different options.
- gegenschall 13y ago
  I do not really understand what you're saying. 1) This is actually modular. I mean... those things are even called "modules". 2) Even if it wasn't modular, how would it matter? You're comparing to the Linux kernel. If that one isn't modular then according to you, systemd also isn't modular. Allright, so be it, did you have any problems with missing modularity in the Linux kernel lately?

Sammi 13y ago

This feels like the Tanenbaum–Torvalds debate all over again.

julie1 13y ago

The major problem is the requirements and the system dependencies it introduces on a core component. http://harmful.cat-v.org/software/dynamic-linking/ http://harmful.cat-v.org/software/dynamic-linking/ systemd introduces a major PITA for the OS packager by putting a lot of dependency on a lot of parts with "moving components" making it both more vulnerable as 1 and more likely to fail (it is not complex it is complicated). Systemd also tends to become a system requirements for a lot of packages making it unsuppressable. It makes the problem of dependency solving logically totally messy without justifying this cost by any real improvement. It is introducing the DLL hell for the most crucial part of the system (PID 1).

peterwwillis 13y ago

I won't ever use a distribution that uses systemd as pid 1. There is only one reason for this. I want to be in control how my system works. To do that I need to be sure of what it is doing and how. As the complexity of a system grows, the number of things it does under the hood grows. At some point I can't keep track of what's going on, and I no longer have confidence in my knowledge of the system, without setting aside time for a code review. With pid 1 being incredibly simple, and the executables it runs being incredibly simple, I know what's going on, I know what the possible problems are and I know how to fix them. The more complex it is, the more difficult it is for me to know what's going on. That's all it comes down to for me. As an aside: A lot of systemd is API-driven, meaning if you want a feature you have to build it into the system as new code. This means you have to be a C developer to leverage some new feature, instead of writing an interpreted script and interacting with it via pipes/plugins/etc. If your current system is dependent on some feature that doesn't yet exist in systemd, you now have to build it in, instead of using what worked with sysvinit before. Not only is systemd broken by design, it's still in its infancy, and does not support the same features that have been built into other systems for years. This additional load of development for many orgs serves no purpose, and is essentially ignored by the leadership that decide to change init systems by fiat.

hdevalence 13y ago
> I want to be in control how my system works. To do that I need to be sure of what it is doing and how. As the complexity of a system grows, the number of things it does under the hood grows. At some point I can't keep track of what's going on, and I no longer have confidence in my knowledge of the system, without setting aside time for a code review. > With pid 1 being incredibly simple, and the executables it runs being incredibly simple, I know what's going on, I know what the possible problems are and I know how to fix them. The more complex it is, the more difficult it is for me to know what's going on. That's all it comes down to for me. How is, say, sysv init better in this regard? Instead of having declarative service files that say what should be done, you have a collection of disparate shell scripts, all with their own implementations of basically the same things. The complexity isn't reduced, it's spread around (and IMO it's more complicated). If you want to have control over your system, wouldn't you want to use an init system that can actually keep track of all of the child processes of a service? Brittle hacks like pidfiles cannot ensure management over all child processes of a service -- this is only possible with cgroups. > As an aside: A lot of systemd is API-driven, meaning if you want a feature you have to build it into the system as new code. This means you have to be a C developer to leverage some new feature, instead of writing an interpreted script and interacting with it via pipes/plugins/etc. If your current system is dependent on some feature that doesn't yet exist in systemd, you now have to build it in, instead of using what worked with sysvinit before. No, you just use D-Bus, from whatever your favorite language is, using that language's D-Bus bindings. Systemd has a stable D-Bus API for this purpose that's guaranteed to work in the future. > Not only is systemd broken by design, it's still in its infancy, and does not support the same features that have been built into other systems for years. Out of curiosity, what features are you referring to specifically? > This additional load of development for many orgs serves no purpose, and is essentially ignored by the leadership that decide to change init systems by fiat. This would be more convincing if it were just Lennart & co. making the decisions, but in the absence of evidence to the contrary, I'm going to go out on a limb and assume that the technical leadership of Fedora, RHEL, OpenSUSE, Arch, and now Debian do not all make decisions by fiat for no reason.

mwcampbell 13y ago

The main systemd binary (version 208 for x86_64 as shipped by Arch Linux) is 1020 KB. I get that the systemd package is really several binaries, but still, does the binary for PID 1 need to have that much in it?

antientropic 13y ago
That's about 1/8000th of the RAM in my laptop. Is 1020 KB really such a big deal?
- mwcampbell 13y ago
  The problem is that software complexity grows exponentially with code size. Well, maybe not exponentially, but more than linearly. And the article established that it's important to keep PID 1 simple.

arca_vorago 13y ago

Stuff like this is why I am increasingly getting back into BSD and recently Minix3. I think this is just part of a larger issue with the *nix ecosystem, where lots of philosophical and design assertions are proving to no longer work, while some are falsely being said to no longer work but still do. With the linux kernel now at over 10 million lines of code, the idea of peer review of source code seems too big of a job for random unpayed volunteers. I think we need to move back to simplicity and decentralization as much as possible, because the problem with centralization is that it makes for a weak system.

brokenparser 13y ago

Systemd makes my life difficult. Case in point: I want to stop cups, so I run "sudo service cups stop". This appears to work, but then it prints out a warning along the lines of: cups was stopped, but it can still be activated by: - cups.sprocket - cups.latch What are these things and why are they there at all? How do I shut them off? It doesn't tell, so stopping a service has become pointless. If I tell it to stop, make it stop!

njharman 13y ago

systemd reminds me of 90's era Microsoft's "release a new service/api/whatever that is almost the same, but different enough to create incompatibility and lockin, include small addition or improvement to encourage adoption, make everything require it to force adoption by dissenters." I'm not joking.