13 ms·
Escape from systemd
- chasil 9y agoA systemd replacement is going to need: - an integrated inetd that can launch non-root processes - ability to chroot() any application at launch - inotify capability (path units) - consistent scripted application startup and control Just about any UNIX could really use an init that can do all of that stuff (even if they don't have cgroups). If you can't, thanks for playing.
- atonse 9y agoI get the impression that OP seems to just be abandoning systemd to build another... systemd. So it seems like not-invented-here syndrome. I don't really know about any of the drama surrounding systemd but I've liked using it so far. It's easily the most attractive feature in Ubuntu 16.04 LTS. Using journalctl for logs is nice because I don't have to keep looking around for where logs are stored and then tail them. But then again, I don't worry about log rotation and things like that. My guess is that journalctl takes care of it. I'm sure I've barely scratched the surface of all the things it can do (like auto-restarting daemons, some security stuff, etc). Am I the exception? Can't be so, if most linux distros have adopted systemd.
- ragebol 9y ago> [systemd] easily the most attractive feature in Ubuntu 16.04 LTS. For me it is as well, I really miss it when working with a Ubuntu 14.04 box. I have not found a single downside to systemd, really.
- usernam 9y agosystemd is not perfect. Debugging systemd is still very hard. There is no way to simulate what actions would be performed to reach a target without actually executing them. There are plenty of issues related to boot and shutdown which are _hard_ to track down because of this. The journal is nice, especially on desktop systems, but it's way too slow to be used in any moderately busy system. The disk space utilization is also _prohibitive_ for archival. We still rely on remote syslog because of this, but it's not without issues: the journal->syslog translation is lossy, and it's compounded with some "quirky" upstream decisions (purely cosmetic) to write unit descriptions in the log as opposed to the real unit name which make tracking down units harder if you don't have the actual journal file. Journal files also get corrupted easily with unclean shutdown. In case of issues (from software to hardware), the local journal quickly becomes completely useless. All in all, systemd is an improvement, but it lacks the debugging tools, performance and overall polish you would expect from a central piece of software such as init. The internal APIs are mostly undocumented or self-referential. The DBus interface is opaque. Things gets muddy very fast as soon as you touch non-central components such as systemd-resolved or networkd. I'm not a detractor, but there are many things systemd can still improve, but it feels we're kind of stuck. I'm quite happy if we have some competition here.
- majewsky 9y ago> Journal files also get corrupted easily with unclean shutdown. Huh? I thought that they introduced this weird binary log format precisely to avoid this kind of error. (I don't remember the exact reasoning, though.)
- usernam 9y agoYou're confusing the binary format with the tamper protection, or file "sealing". The two features are really independent.
- marcosdumay 9y ago"They" were not very successful in this, like they weren't in any other stability related feature.
- cmurf 9y agoThe journal files can get corrupted, but it's a localized corruption that journalctl skips over and continues on without additional data loss. Obviously anything that didn't completely commit is gone, which would be gone anyway with some other format. The main difference is that systemd-journald can a.) become aware of the corruption, b.) warn of it rather than display corrupt info, and c.) recover and continue on. There have been a pile of bugs I've run into and reported where journald gets really confused; e.g. 'journalctl -b-5' won't actually go back five boots, it'll show some other content, or it'll give an error, but using --since will work fine. I haven't seen this problem recently, so the fix may be trickling out.
- bryanlarsen 9y agoThat's not my experience -- once corrupted it stays corrupted. Is this something that's been fixed since Ubuntu 16.04?
- cmurf 9y agoI don't know what you're contradicting because I never said the corruption gets fixed. I said it's skipped over. Specifically what happens is upon detection of corruption, journald stops writing to the corrupt journal file, creates a new journal file, and new entries go there. The utility for reading the journal files, journalctl, can also detect this corruption and will skip over detected corruption and continue on. I have no idea what version of systemd Ubuntu 16.04 has, but in systemd-233 I'm not seeing aforementioned problems with -b switch and corrupt journal files.
- gbromios 9y ago>I get the impression that OP seems to just be abandoning systemd to build another... systemd. So it seems like not-invented-here syndrome. well, he does say: >There may be a bit of NIH syndrome leading to this decision; that’s ok, I can live with that; >I don't really know about any of the drama surrounding systemd I have only been a passive observer of said drama, it boils down to A) init was a stable, widely used, and well understood system that didn't need an update B) systemd takes on far too much responsibility, contrary to the unix philosophy. personally, I have experienced only minor annoyance when having to re-learn familiar commands, though there seems to be a init compatibility layer that has made it pretty transparent to me. It'd probably have been more annoying if I were still an admin.
- lmz 9y ago> init was a stable, widely used, and well understood system that didn't need an update If it didn't need an update why did various parties try and replace it e.g. OSX launchd, Solaris SMF, Ubuntu Upstart?
- gbromios 9y agoI have no idea. Like I said, I was only a casual observer of the rally against systemd. I get the feeling that the opposition was more about ethos than any irrefutable technical argument.
- briffle 9y agoInit also took a VERY long time to start, and had very, very rudimentary concepts of dependencies.
- petre 9y agoLaunchd is actually quite a hurdle to use. XML service files, inconsistent syntax for launchctl commands etc. Apple probably designed it so .plist files would be editable with some GUI tool.
- stuaxo 9y agoI was building a system on raspberry pi involving a few daemons + things that needed starting on powerup + 24h after powerup + before powerdown. It made all these things really easy, I'm not sure why everyone is so keen on doing everything with various bash scripts like we used to.
- JdeBP 9y agoYou are making the oft-made error of thinking that System 5 rc was the precursor to systemd. On several major systems it was not. The precursor was upstart on Fedora and Ubuntu, for example. See http://uselessd.darknedgy.net/ProSystemdAntiSystemd/ http://uselessd.darknedgy.net/ProSystemdAntiSystemd/ for the System 5 rc fallacy and several others to avoid.
- vidarh 9y agoI'm sure there's room for reinventing a better systemd, but most of the attempts I've seen either fail to understand and intentionally ignore just how many things systemd simplify. I can understand wanting to decouple and simplify that, but I think a lot of the time the people trying to reinvent it do not understand the concerns that systemd solves for people. E.g. a typical issue is process restarting. Simple. Can be done in a couple of dozen lines. Except that solves nothing for most people: If your server is spinning in place trying to start your big-proper-process-monitor, and it's failing to log anything because rsyslogd or whatever is also just spinning in place, and the boot processes that should have mounted the filesystems needed to get things working is repeatedly crashing? You can mitigate all of that, but now you're creating a separate mechanism (maybe as simple as writing to disk, but still) for early-stage logging, and a separate mechanism and your init needs to deal with things like throttling restart and handling them and so does a separate process manager, and so on. In order to simplify things, there's duplication of functionality all over the place. A huge part of the architecture of systemd is down to starting to pull at that thread and untangling all of these issues. Is it perfect? No. Is it the solution to all of the concerns they've uncovered? Probably not. But what we had before was so much worse, and I'm so happy someone have had a go at addressing them. And frankly, it's great that animosity towards systemd has motivated so many people to finally try to address this issues in other ways. Maybe one of them eventually solves what none of them did before systemd forced peoples hand.
- captainmuon 9y agoI think for many people who are dissatisfied or angry, systemd didn't simplify things. Many people have simple use cases, and for them it made things much more complicated. Monolithic, incomprehensible, unintrospectable. It is like a modern car engine which is just a sealed black box under the hood, full of electronics. Sure, it works great, it is progress compared to before, everything is there for a reason. But you can't easily fix things yourself. And for many people, the old kind of engine was good enough. I mean, for my simple uses, what is wrong with `sudo service nginx restart`, or `sudo killall nginx`? For many people, it is really dead simple. I get that it is not simple for everybody, but that is why we should have choice, not one-size-fits-all.
- captainmuon 9y agoThat's fine to me. Linux is about choice. One of my biggest problems with systemd is that other stuff starts to depend on it. For things like GNOME, it is not just an init system, it is THE one init system. Also, I'm sure journald is useful, but I've had so many problems with it. For me as a Desktop and occasional remote computing, small server user, logs are ephemeral. Something goes wrong, I can quickly grep the logs which are plain text. I didn't really feel any advantage from the modern, binary, structured logs. Also, I spent hours installing something on WSL (Linux on Windows), finding that everything works except for the systemd logs, which are empty. The command that controls services spits out hundreds of cryptic lines, services interweaved with all kinds of other objects, when I just want to list the running services. It has too many moving parts for my taste. I don't need socket activation. I don't need a separate DNS resolver in my init system (unrelated note: I'd love more command line/GUI tools to be able to use a separate DNS resolver, work on separate interfaces, ...!), and so on. Even if I will never use this guy's init system, if it is even a bit successful it will mean that people will stop assuming systemd, leading to a looser coupling, more interface stability, easier porting between systems, and more choice. So I think it's a great thing he is trying.
- lima 9y ago> I don't need socket activation. I don't need a separate DNS resolver in my init system It's. Not. No idea why people keep repeating this. The DNS resolver belongs to systemd-resolved, is entirely optional and off by default. Nothing about it is in the init system. And maybe you don't need socket activation, but there are plenty of valid use cases for it. See: http://0pointer.de/blog/projects/socket-activation.html http://0pointer.de/blog/projects/socket-activation.html
- mook 9y ago"systemd" is the init system, therefore anything that has it in the name is part of that init system. See also journald, which nobody mentions as being part of the init system even though it's more tied to it (is it even possible to have systemd without journald?) If they want to change that perception, they will need to change the name (and hopefully stick a stable ABI around it, but that's unlikely to ever happen).
- lmm 9y ago> Am I the exception? Can't be so, if most linux distros have adopted systemd. By that logic why use Linux in the first place when Windows is so much more popular? If it's anything like other projects from the same source systemd is great when it works and impossible to debug when it doesn't. Which is exactly what I came to OSS to get away from.
- phil21 9y agoThe basic summary is basically: Developers love systemd, it's much more "developer friendly" and the way it's laid out makes sense to them (e.g. your love of journalctl, which is bizzaro-world for me). No more thinking about ugly operating systems and those silly details! Old school sysadmins and those who actually get their hands dirty with Linux systems under the hood despise it for what it is. It's the windowsfication of Linux, in my - and many others - opinions. A giant PID 1 that controls everything, re-implementing such basics as ntp and rm, is imo entirely anti-unix-like. Systemd is great until it breaks. Once it breaks though, good luck. They've re-invented the wheel multiple times over, and as time goes on more and more things become part of monolithic systemd. If you do happen to find a weird bug, prepare to get yelled at and be forced to vigorously defend your bug report. The joke in the office is when Pulseaudio support gets added.
- mateuszf 9y ago> A giant PID 1 that controls everything Kernel is also a giant code that controls everything. Is this such a bad thing?
- VLM 9y agoYes. There's been a lot of theoretical and practical work around the last couple decades about putting as little as possible in your single point of technical and legal/patent failure. Small tools that do as little as possible, but do it very well, composed into bigger systems. Philosophically identical to RISC vs CISC.
- Twirrim 9y agoThe kernel is a component that has a straightforward purpose on a system, and interferes as little as possible with the actual usage of the system. You could describe it as "It gets the job done, and stays the hell out of the way" Even with the kernel there is an element of choice about what parts of it you do / don't use, e.g. you can use ext4, XFS etc. etc. and even have the choice of providing kernel modules so you can bolt on ZFS. The kernel doesn't start to dictate who or what software does DNS resolution, for example. systemd keeps taking over more and more control of various aspects of your system and effectively removing choice by making it hard to use alternatives because it starts to make assumptions based on the inbuilt versions. systemd is presenting you with a "one-size-fits-all" scenario. The problem is one size never fits all. One of the biggest strengths within the *nix ecosystem has been the fundamental understanding no two use cases are entirely the same, what works well for one purpose doesn't work well for others, and choosing the right software to meet those various purposes is invaluable. It's impossible to make software that is efficient for every single use case. The freedom and possibility to choose the right tool for the right task is essential to designing and building new, innovative, or even just plain old efficient systems.
- dwc 9y ago> abandoning systemd to build another... systemd Well, systemd is a particular answer to a set of problems that have been well known for a while. Building an alternative solutions to those problems is not necessarily "another systemd". > drama surrounding systemd The quick summary: many people are critical of the architecture and design choices and implementation details in systemd in particular, rather than that the underlying issues it's supposed to address are real. > but I've liked using [systemd] so far This isn't surprising, as some of the problems it addresses are real, and a pain. But there are other implementations that also address many of the same issues. Whether those would make you as happy as systemd is another matter.
- Omnius 9y ago"I have at least briefly looked at a number of them." So you are wanting to build a new init system from scratch and you haven't taken the time to see all the stuff that already exists? I mean if this is just a project you want to do go for it but if you are seriously trying to build something for others you know whats out there and why people choose one over the other beyond just systemd and sys V.
- JdeBP 9y agoThis is a common fault. Not knowing the history and what has already been done long since is an error exemplified in this discussion on this very page, in fact. (One place to start learning from is http://blog.darknedgy.net/technology/2015/09/05/0/ http://blog.darknedgy.net/technology/2015/09/05/0/ .) But in fairness Davin McCall has done more than most. There is a short writeup of xyr opinions of various systems, that comes with the Dinit source code. It gets a number of things wrong about several of the systems and does not go into as much depth as, say, the Debian Technical Committee members did. But it does show that M. McCall has at least looked, just as xe states.
- marcosdumay 9y ago> So you are wanting to build a new init system from scratch and you haven't taken the time to see all the stuff that already exists? Honestly, my impression is that this is how everybody that did something widely adopted acted. I don't think it's desirable, bu there may be a problem with taking your time to review everything out there.
- deleted 9y ago[deleted]
- Mister_Snuggles 9y agoOne thing I've learned to accept about Linux, in general, is that it likes to go its own way relative to other Unix-like systems. I've come to accept systemd and actually like it in the sense that, for the most part, it just works and gets out of my way. The more I learn about journalctl's options for viewing logs, the more I like it. The case of Linux going its own way that drives me nuts though is the replacement of 'ifconfig'. The replacement seems to be in progress, with some distributions shipping 'ifconfig' in addition to the 'ip' and others only shipping 'ip'. EDIT: Other Unixes have also gone their own way with init systems to meet their own needs. MacOS, for example, replaced initd with launchd. Solaris, way back in 2005 with Solaris 10, replaced init with SMF[0] which does some of the things that systemd does on Linux. [0] https://en.wikipedia.org/wiki/Service_Management_Facility https://en.wikipedia.org/wiki/Service_Management_Facility
- xenadu02 9y agoA word of warning: launchd has been rewritten more than once. People claiming systemd is some slapped-together piece of junk with bad architecture are probably wrong. People who think they can write a better systemd on their first try are probably wrong. This stuff is really hard.
- justinsaccount 9y ago> One thing I've learned to accept about Linux, in general, is that it likes to go its own way relative to other Unix-like systems Solaris is Unix. Solaris shipped SMF over a decade ago. OS X is Unix. OS X shipped launchd in 10.4, back in 2005.
- Mister_Snuggles 9y agoIn retrospect, "Going its own way" for init systems is a normal thing in the Unix world. That was actually a terrible example. At best I could say that the sheer scope of systemd is an example of Linux going its own way. SMF and launchd, for example, don't change how logging is done. I don't believe either of those include their own DNS resolver either.
- nithinm 9y agowhy the c++ hate in comments? because of the memory leaks?
- oneplane 9y agobecause of the hidden magic. Linus has a great message on a mailing list somewhere on this. Basically it boils down to: don't have hidden magic in important OS parts, it's bad and helps nobody except for people that explicitly want to use c++.
- Santosh83 9y agoThe crux of the issue I believe is C++ can lead a bewildering mess of coding styles and abstractions, making it difficult for the thousands of people working on the kernel to grasp each other's code. This would be mitigated to a large extent by using the simpler C, hence the rule. C++ needs a lot more discipline and thought if you don't want unmaintainable code years down the line. Ironic given that abstractions introduced by C++ are partly to make code more efficient to write, extend and maintain.
- antientropic 9y agoWhat "hidden magic"?
- marcosdumay 9y agoA C++ line does way more things than a similar looking C line. The developer may not be aware of all those extra things.
- deleted 9y ago[deleted]
- oneplane 9y agoI wonder why there is no bigger adoption of launchd. It's open source and works fine. Plenty of BSD's have it available for PID1 usage, and it doesn't have the feature creep most init replacements seem to grow on.
- Mister_Snuggles 9y agoIt looks like the FreeBSD folks have been doing some work[0][1] in that area. Sadly, it doesn't seem like it's actively being worked on at the moment. FreeBSD's init system works well enough, but there's no question that it could be better. [0] https://wiki.freebsd.org/launchd https://wiki.freebsd.org/launchd [1] https://github.com/freebsd/openlaunchd https://github.com/freebsd/openlaunchd
- VLM 9y ago"FreeBSD's init system works well enough, but there's no question that it could be better." Problems or features? I haven't run in to any problems. From a feature perspective I've run into the system policy bug where the name of a package has a minimal relationship with the name of the file read in /etc/rc.conf.d... For example, say you're installing the package named "isc-dhcp43-server", you need to add a file named /etc/rc.conf.d/dhcpd. Why not a file named something "obvious" like /etc/rc.conf.d/isc-dhcp43-server ? Sometimes the names are close enough to be Extremely annoying, like package name "logstash-forwarder" and /etc/rc.conf.d/logstash_forwarder. Obviously thats not a bug in the init system, thats a bug mis-using the init system. Or is that mis_using? Or misusingd-4.2? Its the kind of issue thats super annoying one time when you write the ansible script and then you forget about it until next time you're writing an ansible script to do something else. I have not run into any lack of features problems with freebsd init. Although I admit to having run some end user applications (not system stuff) using supervisord because the end users wanted a pretty web interface for restarting and logs and stuff.
- Mister_Snuggles 9y agoThe only 'problem' I've had with FreeBSD's init was when an NFS mount was unavailable that the machine hung during boot. The mount didn't have anything required for boot on it, but there were services that depended on those mounts being there. In retrospect, I probably should have added "failok,-t=10" to those fstab entries. This one was definitely my fault. An ideal init system would have known (or been taught) that services x, y, and z depend on NFS mounts /a, /b, and /c, so the system could have started mounting those in the background and let other services (like sshd!) start up as required. As to your point about the service names, that bugs me as well.
- Sir_Cmpwn 9y ago>I'm making an init system Awesome, maybe I won't have to! >C++ Whelp, nevermind.
- peterwwillis 9y agoYou can skip reading this article, it isn't even finished being written, and is more of a product announcement for a systemd alternative in C++.
- quigonjinn 9y agoThere's also GNU Shepherd.