3 ms·
I have way too many cases of systemd startup items hanging with their timeout for no good reason. Worse, my most complicated high-uptime machine usually does n
by cracauer 7y ago
I have way too many cases of systemd startup items hanging with their timeout for no good reason.
Worse, my most complicated high-uptime machine usually does not shut down in a reasonable time. Systemd says "waiting for session user cracauer" (something like it) for whatever reason. It also hung on undoing swapspace, when that swapspace was a custom stack of block layers. I don't need swapspace to be "shut down". It went into 20 minutes something always stating another timeout when one expired. I mean, WTF?
The problem here is that I had to do an unclean shutdown on that machine (reset button) multiple times, and that I really can't have.
This also illustrates a point I have been making about Linux and the BSDs from day one: When the BSDs boot or shut down the keyboard is connected to the rc scripts. You can do Control-C for SIGINT and Control-\ for SIGQUIT, The latter makes everybody in the stack leave a coredump on disk.
Now, compare these two:
1)
in BSD when there is a startup or shutdown item taking too long or hanging I can abort that single item and I can later debug what was going on with the coredumps.
2)
on a Linux system with systemd I cannot "cut through" startup or shutdown items I don't want to wait for, and there is no way to debug any of that if the machine doesn't actually come up completely, or if thing happen at shutdown. The only interaction I can conduct is the reset button.
To add insult to injury, systemd also hides a lot of error messages that would traditionally be dumped on the console, and instead of giving me that error message systemd captures it and print on the screen instructions how to get that error message (which usually are longer than the error message was, WTF?). Instructions that I cannot follow because somebody detached my keyboard, and because I will never be in that machine in an up state while that context is available.
This also invalidates the point that systemd could be one platform that you learn once and then use on a wide basis. The debugging abilities are nowhere close to adequate, so you have to learn by try-n-error.
- webstrand 7y agoI've struggled with, and am still struggling with, units hanging on shutdown. It's my biggest complaint with systemD aside from journald eating its own logs. I've been using magic SysRq to force hanging units to quit. SysRq + e sends SIGINT to all processes, and SysRq + i sends SIGKILL.
- cracauer 7y agoSending SIGINT to all processes isn't what is needed here. It needs to go to the "processes" that are currently going on. With systemd's parallel execution that can still be narrowed down in a useful way, especially is some tail of some hanging command is running alone by now. Obviously what the BSDs are doing, sending SIGQUIT to just the currently blocking-the-shutdown processes, is far superior. Now the coredumps are floating around the filesystem and they'll be there after reboot, and you can see who did what and was hanging in what backtrace.
- AsyncAwait 7y agosystemd errs on the safe side and thus it tries really hard to shut things down safely vs just yanking the plug. The timeouts are easily configurable if that's a problem for you, but I'd argue that if services hang too often for you, there's an underlying problem with your system somewhere, (i.e. your mounts or such, I'd consult journald for dependency cycles).
- cracauer 7y agoThere might be some way in which I had run a system that caused those endless slowdowns. But the point is, and that's why I think systemd is garbage, that there is nothing there that helps me debug what is going on. If the shutdown is hanging there is nothing to do except the reset button or waiting for an eventual reboot. In neither case will there be any forensics about what exactly systemd had tried to do and how it thought that it failed. That is the part that is not acceptable. P.S. I shouldn't have to manually adjust a timeout on disabling swapspace on shutdown to prevent a need for a reboot by reset button. The kind of thinking that got big timeouts into those things is precisely why I say that systemd people just think differently than I do.
- AsyncAwait 7y ago> that there is nothing there that helps me debug what is going on That's not really true. Journald is your friend. I find many of the complaints about systemd stem from unfamiliarity with its tools. The Arch Wiki gives a really nice overview of the most common usage scenarios[1] & [2], that's worth investing some time into. Also [3] is your friend. Just refusing to put any time into learning systemd and then complaining about it, is one approach, but not a particularly productive one. In your case, I'd try something like `systemd-analyze verify default.target` and go from there. Additionally, look into /etc/system.d/system.conf, specifically adjusting the Timeout values to a less conservative value. 1 - https://wiki.archlinux.org/index.php/Systemd https://wiki.archlinux.org/index.php/Systemd 2 - https://wiki.archlinux.org/index.php/Systemd/Journal#Filtering_output https://wiki.archlinux.org/index.php/Systemd/Journal#Filteri... 3 - https://www.freedesktop.org/software/systemd/man/systemd-analyze.html https://www.freedesktop.org/software/systemd/man/systemd-ana...