5 ms·

This anti-systemd meme is getting tiring. I generally find it is perpetuated by people who use Linux at a hobby level. Rarely do I speak with guys in Ops who di

by pard68 7y ago

This anti-systemd meme is getting tiring. I generally find it is perpetuated by people who use Linux at a hobby level. Rarely do I speak with guys in Ops who dislike systemd. I am not trying to discredit people here, there are hobby users who are more knowledgeable about Linux related systems than many guys who are professionals. My point is that Linux and systemd are both meant for use on servers, in production, where speed and consistency rule. I manage 3000 Linux servers and 20,000 services running on those servers. When I log into a machine and am welcomed by a RHEL6 motd I groan, because sysvinit is not enjoyable to use, it takes longer to reboot those machines, and tracking down logs is a chore. When I log into a RHEL7 machine I am not elated, but I am glad to know that pid 1 isn't trying to make my life harder.

syn0byte 7y ago
Hi, ops guy; Fuck systemd. Boot time: A possible 5-10 extra seconds from sysvinit is the problem here not the 5 minutes of BIOS/EFI POST time for those server boxen? Logs: Where's the storage of the log data in journalctl and what tools can I use to read that data in a small PXE rescue environment? But hot damn I didn't have to ls /var/log so that's nice. Sarcasm aside how does it make your life any easier? I got a stack of FIXMEs to work on to get our infrastructure working, lot alone stable, on systemd.
- pard68 7y ago
  > A possible 5-10 extra seconds > the 5 minutes of BIOS/EFI POST time Yeah we measure boot up in seconds. If I restart a machine and it takes more than ~10 seconds to boot I send a ticket to our infra guys to figure out why the server is configured poorly/troubleshoot the slow start. >Logs I guess I prefer `journalctl -u service.service -f` or `journalctl -p "crit" --since "2 days ago"` to the ole `ls /var/log` and then `grep`ing Honestly, I don't understand how it makes life harder. To me, systemd is a huge QoL impovement. Even in the most mundane tasks like `systemctl restart tomcat-*`
silversconfused 7y ago
Being dismissed as a meme is tiring as well. I like hacking on my system at boot time, and dismissing my use case because of your own is kinda hostile.
- teddyh 7y ago
  The anti-systemd group contains vocal trolls, memers, and ranters, so anyone who’s on same side will have to live with being grouped with them.
- pard68 7y ago
  If you are referring to my comment regarding Linux not being for desktops... I use Linux on my desktop. The only non-Linux machines in my house are my work Windows machine and my sister in law's Windows machine. All my personal machines, my wife's laptop, and my kids' desktop are all Linux flavors. Linux is a great desktop imho. That said, Linux is not primarily a desktop machine. It gets some pro-Desktop patches but at the end of the day, Linux is designed for servers and paid for by people interested in its usage as a server.
peterwwillis 7y ago
Ops person here. When you set up a node, you set up all its networking, its services, its packages, its configuration, etc. It doesn't matter what your technology is - you set it up to do whatever it needs, and then hopefully you never have to re-do that again. Systemd's hydra of services affected a lot of that, so we had to learn all the weird gotchas to get newer distros to work properly. And it's not just a little different, it's more complex. This increases the number of bugs, the time to test and roll out changes, integration work, etc. So the process is more complex. And the operation is also more complex, meaning more training for anyone who's used to how things worked for 30 years. Doing minor ops work now requires looking at 5 different manuals, requiring extra software, adding configuration management where it wasn't before, changing support instructions, healthchecks/automation tasks, migrations to newer disk images while supporting legacy functionality, etc. And here's the part that really kills me. Alternatives to all the parts of systemd already existed; if you really needed a better system, you could pick up any similar software and make it work! But then distributions start rolling this out en-masse for all new releases, and we don't have a choice to opt-in. It was a breaking change that did nothing for us that we didn't already do for ourselves, even requiring more work than before. Do our systems boot faster now? My EC2 instances still seem to take forever to spin up, and physical machines still take ~5 minutes to POST, so I'm going to say no. Can we do new things that we couldn't before? No. Are our systems more stable now? No. Are they less complex? No. So while some people talk up the theoretical benefits, I as an ops person have not seen them. But I still have to look up man pages to remember the three different filesystem trees systemd keeps files in, or the wonky configuration files, or what 5 extra commands I need to get something done. If you're going to make breaking changes, at least have it make my life easier after I get used to it. Systemd has not done that. And it seems to just keep getting heavier.
- pard68 7y ago
  Our bottle neck is often pid 1 for boot time. We don't do mem check on boot so that saves us 1 second * Gb_of_RAM in boot, which is why most BIOS take forever on a server. We also don't run anything in PCI slots so we have the PCI checker disabled too (all our servers are Dells they have some software you can license to config the BIOS). I'm new to the game, I only vaguely remember a time when Ubuntu wasn't on systemd. I don't have 30 years of experience with any init system, but I do have 10+ years with systemd. I didn't use another init until I got this job and had to learn sysv. So I'm coming from the exact opposite, it sounds like, place of you. To me, everything you said is 100% true, but about sysv and its line. I also don't provision too many servers these days because with the exception of a few things that were deemed "to important" all our servers have automated build scripts to go from 0 to prod.
asteven 7y ago
My experience is like 100% different from yours. I have the feeling systemd is meant exactly for what you call hobby or novice users. I actually enjoy some of it on my laptop to be honest ;-) But in the datacenter it has caused me hours and hours of 'fixing problems that did not exist before systemd came along'. Most of it's 'improvements' target the desktop user. Many of it's 'improvements' actually get in the way in the server room. I mean, why would I want to optimize my boot time on a server with all this parallel startup crap? The server spends 10 times more time in BIOS and POST then it takes to boot the OS. It has always been like this. And it still is like this. From a config management point of view I like systemd. I gives me a consistent API to enable/disable start/stop services. That's awesome. I also like that I can 'override/patch' upstream units without having to maintain a fork. Perfect. But many other things that came from the systemd ecosystem are questionable. E.g. systemd-resolved, timesyncd and that kind of stuff. Maybe these are nice for desktop users who want a click and play experience on their laptop. No problem with that. But why do they have to break my server's for this? I mean seriously: Linux on the desktop is irrelevant. Why optimize the whole OS/init for a irrelevant use case?
- pard68 7y ago
  Boot time is a big deal for us. Not sure if it matters, but I don't get paid enough to decide what matters. All I know is we paid extra for some Dell software that lets you fine tune the BIOS and we streamline for boot speed. We're currently in the process of moving anything that can be run on a VM to a VM so this will be even less of a concern in the future and pid 1 will be more of a bottle neck. That said, except for the db servers, we don't run mem checks on boot so our servers are very fast to boot (I believe Dell cites that each Gb of RAM is equal to 1 sec. of boot time when you are running with the mem checker on). The only time I ever do a "deep dive" and manual setup a server is when I have to provision a server for a new department/service. Generally speaking ansible does the work for me and I just hit go on my console and then go get a coffee. So I don't see much of a headache in server provisioning. I also am on the automation side of Ops so I do a lot more work with a already setup server, and in that case, like you said, systemd API is a godsend.

by pard68 7y ago

syn0byte 7y ago

Hi, ops guy; Fuck systemd. Boot time: A possible 5-10 extra seconds from sysvinit is the problem here not the 5 minutes of BIOS/EFI POST time for those server boxen? Logs: Where's the storage of the log data in journalctl and what tools can I use to read that data in a small PXE rescue environment? But hot damn I didn't have to ls /var/log so that's nice. Sarcasm aside how does it make your life any easier? I got a stack of FIXMEs to work on to get our infrastructure working, lot alone stable, on systemd.

pard68 7y ago
> A possible 5-10 extra seconds > the 5 minutes of BIOS/EFI POST time Yeah we measure boot up in seconds. If I restart a machine and it takes more than ~10 seconds to boot I send a ticket to our infra guys to figure out why the server is configured poorly/troubleshoot the slow start. >Logs I guess I prefer `journalctl -u service.service -f` or `journalctl -p "crit" --since "2 days ago"` to the ole `ls /var/log` and then `grep`ing Honestly, I don't understand how it makes life harder. To me, systemd is a huge QoL impovement. Even in the most mundane tasks like `systemctl restart tomcat-*`

silversconfused 7y ago

Being dismissed as a meme is tiring as well. I like hacking on my system at boot time, and dismissing my use case because of your own is kinda hostile.

teddyh 7y ago
The anti-systemd group contains vocal trolls, memers, and ranters, so anyone who’s on same side will have to live with being grouped with them.
pard68 7y ago
If you are referring to my comment regarding Linux not being for desktops... I use Linux on my desktop. The only non-Linux machines in my house are my work Windows machine and my sister in law's Windows machine. All my personal machines, my wife's laptop, and my kids' desktop are all Linux flavors. Linux is a great desktop imho. That said, Linux is not primarily a desktop machine. It gets some pro-Desktop patches but at the end of the day, Linux is designed for servers and paid for by people interested in its usage as a server.

peterwwillis 7y ago

Ops person here. When you set up a node, you set up all its networking, its services, its packages, its configuration, etc. It doesn't matter what your technology is - you set it up to do whatever it needs, and then hopefully you never have to re-do that again. Systemd's hydra of services affected a lot of that, so we had to learn all the weird gotchas to get newer distros to work properly. And it's not just a little different, it's more complex. This increases the number of bugs, the time to test and roll out changes, integration work, etc. So the process is more complex. And the operation is also more complex, meaning more training for anyone who's used to how things worked for 30 years. Doing minor ops work now requires looking at 5 different manuals, requiring extra software, adding configuration management where it wasn't before, changing support instructions, healthchecks/automation tasks, migrations to newer disk images while supporting legacy functionality, etc. And here's the part that really kills me. Alternatives to all the parts of systemd already existed; if you really needed a better system, you could pick up any similar software and make it work! But then distributions start rolling this out en-masse for all new releases, and we don't have a choice to opt-in. It was a breaking change that did nothing for us that we didn't already do for ourselves, even requiring more work than before. Do our systems boot faster now? My EC2 instances still seem to take forever to spin up, and physical machines still take ~5 minutes to POST, so I'm going to say no. Can we do new things that we couldn't before? No. Are our systems more stable now? No. Are they less complex? No. So while some people talk up the theoretical benefits, I as an ops person have not seen them. But I still have to look up man pages to remember the three different filesystem trees systemd keeps files in, or the wonky configuration files, or what 5 extra commands I need to get something done. If you're going to make breaking changes, at least have it make my life easier after I get used to it. Systemd has not done that. And it seems to just keep getting heavier.

pard68 7y ago
Our bottle neck is often pid 1 for boot time. We don't do mem check on boot so that saves us 1 second * Gb_of_RAM in boot, which is why most BIOS take forever on a server. We also don't run anything in PCI slots so we have the PCI checker disabled too (all our servers are Dells they have some software you can license to config the BIOS). I'm new to the game, I only vaguely remember a time when Ubuntu wasn't on systemd. I don't have 30 years of experience with any init system, but I do have 10+ years with systemd. I didn't use another init until I got this job and had to learn sysv. So I'm coming from the exact opposite, it sounds like, place of you. To me, everything you said is 100% true, but about sysv and its line. I also don't provision too many servers these days because with the exception of a few things that were deemed "to important" all our servers have automated build scripts to go from 0 to prod.

asteven 7y ago

My experience is like 100% different from yours. I have the feeling systemd is meant exactly for what you call hobby or novice users. I actually enjoy some of it on my laptop to be honest ;-) But in the datacenter it has caused me hours and hours of 'fixing problems that did not exist before systemd came along'. Most of it's 'improvements' target the desktop user. Many of it's 'improvements' actually get in the way in the server room. I mean, why would I want to optimize my boot time on a server with all this parallel startup crap? The server spends 10 times more time in BIOS and POST then it takes to boot the OS. It has always been like this. And it still is like this. From a config management point of view I like systemd. I gives me a consistent API to enable/disable start/stop services. That's awesome. I also like that I can 'override/patch' upstream units without having to maintain a fork. Perfect. But many other things that came from the systemd ecosystem are questionable. E.g. systemd-resolved, timesyncd and that kind of stuff. Maybe these are nice for desktop users who want a click and play experience on their laptop. No problem with that. But why do they have to break my server's for this? I mean seriously: Linux on the desktop is irrelevant. Why optimize the whole OS/init for a irrelevant use case?

pard68 7y ago
Boot time is a big deal for us. Not sure if it matters, but I don't get paid enough to decide what matters. All I know is we paid extra for some Dell software that lets you fine tune the BIOS and we streamline for boot speed. We're currently in the process of moving anything that can be run on a VM to a VM so this will be even less of a concern in the future and pid 1 will be more of a bottle neck. That said, except for the db servers, we don't run mem checks on boot so our servers are very fast to boot (I believe Dell cites that each Gb of RAM is equal to 1 sec. of boot time when you are running with the mem checker on). The only time I ever do a "deep dive" and manual setup a server is when I have to provision a server for a new department/service. Generally speaking ansible does the work for me and I just hit go on my console and then go get a coffee. So I don't see much of a headache in server provisioning. I also am on the automation side of Ops so I do a lot more work with a already setup server, and in that case, like you said, systemd API is a godsend.