19 ms·

Spotify position in support of systemd in the default init debate

234 points by possibilistic 13y ago

mwcampbell 13y ago
Just saw this interesting post by Noa at Spotify (speaking on his own behalf) later in the thread, about the trade-off between monoculture on the one extreme, and effort spread too thin between competing solutions on the other: https://lists.debian.org/debian-ctte/2014/01/msg00306.html https://lists.debian.org/debian-ctte/2014/01/msg00306.html
beefhash 13y ago
I don't understand how this is still anything to discuss. systemd is the only option; GNOME is already using it for some parts and will probably not hesitate to integrate it even more deeply to push Poettering's agenda. Whether or not the sysadmins want to learn systemd and regardless of whether it's the technically superior super-init system (you can't really call systemd an init system anymore), it's going to be forced down our throats anyway. What's with the holdup?
- Nursie 13y ago
  GNOME is far from the only FOSS desktop these days. A desktop that demands you have a specific init system is not onw I care to use.
- bkor 13y ago
  Systemd provides various things that desktop environments want to use. This is not limited to GNOME. I talked to Enlightment and they're going to make more use of various bits (e.g. user session). Similar with KDE. Dismissing it entirely like you're doing and pretending this is about forcing: good luck with that. Just like distributions were forced to use it right? Not merit, just pushed? Try coming up with something concrete. Until that the lack of specifics in your posts tells me enough.
dinkumthinkum 13y ago
The person that wrote is a "Free Software ombudsman" ... Pretty serious stuff we got going in software now. :)
trump 13y ago
Keep in mind, this is coming from the same company that gives their devs root: https://www.youtube.com/watch?v=pts6F00GFuU#t=169 https://www.youtube.com/watch?v=pts6F00GFuU#t=169 Otherwise, I agree with their endorsement of systemd.
bitwize 13y ago
Systemd is the Linux init system. Using anything else at this point is folly. This debate is a tempest in a teapot stirred up by whiners and haters who are resisting what the community overwhelmingly supports as a technically superior, easier, and designed-for-Linux init system.
- akira2501 13y ago
  It's better than SysV init, but there are a dozen other init's out there, all with different properties. Not everyone cares about "linux on the desktop," and not everyone thinks that freedesktop.org is doing the best job as far as that goes either.
  - bitwize 13y ago
    It's better than SysV init, but there are a dozen other init's out there, all with different properties. And none of them leverage Linux's features as extensively as systemd. By the Linux init system, I mean the one overwhelmingly supported by most of the Linux community, and the one which is best designed to work with Linux. Not everyone cares about "linux on the desktop," and not everyone thinks that freedesktop.org is doing the best job as far as that goes either. Who's talking about desktops? Systemd is not a freedesktop project, and it was designed to make server administration easier by offering much finer grained control and monitoring of system services with a much better interface. Many of its most controversial features -- such as binary log files -- don't make sense except in a server context where the tampering-attestation features of systemd-journald make it a far more secure solution than plain text logs. You should really read Lennart's essays and blog posts about systemd to get a clearer picture of its tremendous advantages. Whatever you do, you'd best get used to systemd. It has all but won.
    - akira2501 13y ago
      I'm not trying to be snarky here; but systemd is hosted on freedesktop.org, is it not? That is its primary place of development as well, no? Admittedly, I've always associated systemd as a project as part of this nebulous "effort" to bring linux on the desktop forward. I also assumed that's why they went to the trouble to integrate systemd into dbus so heavily. I have read some of Lennart's essays -- but I generally strongly disagree with his conclusions, so I haven't been paying attention for a while. I think his solutions are over complicated and rely on too much impenetrable technology to actually save me any time. The arguments have been made elsewhere, so I won't go into it that deeply, but I think departure from the "unix way" is folly and will only make administration more difficult. As far as getting used to it goes; this is still unix. I can still install whatever initd I like after the fact. Which I do. Yes, this means I have to do more work than I would have to do if I relied on vendor packages, but we generally use custom packages for all our critical services anyways. Since we're going to have to deal with it whether or not we're using systemd, and we're already rolling out a non-standard initd to our distributions, we don't feel we have to make any investment in learning systemd.
myrandomcomment 13y ago
Both systemd and upstart have added a level of complexity I would prefer to avoid. The simplistically of sysvinit is greatly missed. But then again I am an old fuck so..
- bkor 13y ago
  The complexity is there, you just don't see it. Systemd handles all types of services. Sysvinit does not. Systemd abstracts a lot, making it easier per service (by putting more in systemd). In sysvinit a lot is copy/pasted across services. IMO making everything more error prone.
mmagin 13y ago
What made me hate upstart was that it still is willing to get the default runlevel from /etc/inittab, but it doesn't bother to emit warnings to the user that the rest of /etc/inittab is completely ignored, at least on the version of CentOS I was using. I think it's particularly user-hostile design for any program to accept configuration that does nothing (excepting whatever comment syntax a particular config file uses.) I would vastly prefer if it either made lots of complaints about the ignored lines in inittab, or even if it it failed hard -- because then I would have to fix the problem immediately. I have used systemd, but never actually poked around at its configuration (having just run Debian with it as a desktop user), so I can't properly judge it yet.
deleted 13y ago
[deleted]
- quarterto 13y ago
  Call me old school but I don't see why we can't just keep compiling everything from source. If one package has a dependency on another, just check if the other package is built or refuse to build. What is driving this need for "modern package management" ?
  - keithpeter 13y ago
    Did a custom kernel under RHEL beta on this Thinkpad x61s core duo 2 laptop. Took 3 hours and used 16Gb of hard drive space. Compiling R takes about 20 minutes. Binaries are quicker.
  - yogo 13y ago
    Because it's a time wasting process? Unless of course, everything you need to compile is relatively small.
    - _delirium 13y ago
      That's been my experience with MacPorts, which does have some binary packages, but much lower coverage than a distribution like Debian, so fallback building from source is common. It can be really annoying when you just want to install some utility and it takes literally 3 or 4 hours to install, because somewhere in the dependency or build-dependency chain is some big and slow-to-compile thing like GTK, X.org, or a newer GCC.
      - Zancarius 13y ago
        Gosh, you're reminding me of those days of compiling KDE or Firefox on an old box under Gentoo. I think I still wake up in a cold sweat from time to time. :)
    - Zancarius 13y ago
      Exactly. Time constraints and laziness are my two primary reasons for preferring a package manager over source compilation. That's not to say a good sysadmin shouldn't know how to compile from source--you should--but building every package from source and wasting time on a system with a strong package manager almost seems criminal or at least disrespectful to the people paying for that time. Package managers are a time saving device, and these days they're generally very good. If you have a need that isn't met by a modern package manager, I can't help but think it's an exceptionally niche use case. I used to compile many of my packages from source because of tunings that weren't present in the pre-built packages or pre-built packages that weren't available/out of date (this was probably on FreeBSD some 10+ years ago, although I'm reluctant to include building from ports in this because that is mostly automated and doesn't really count). But when I migrated to Gentoo where builds are automated (and then to Arch) and later experienced the Debian/Ubuntu side of things, I began to appreciate what a colossal waste of time it was to dig through configure options, set flags, etc., much less doing everything else manually. It's just more convenient to allow the platform itself to take care of things or offload worrying about dependencies to the package maintainer who probably knows more about building these packages than I do (or care to). Again, it's not to disparage the value of building from source (or knowing how), but I have other things I'd rather put my time into, like whining about compiling from source. ;)
cakeface 13y ago
I highly recommend that anyone interested in systemd who doesn't know much about it read Lennart Poettering's initial descriptions of it. I knew very little about Linux init systems and I found this incredibly interesting and informative. http://0pointer.de/blog/projects/systemd.html http://0pointer.de/blog/projects/systemd.html
- pingswept 13y ago
  Thanks for that. Very interesting read.
deleted 13y ago
[deleted]
Shish2k 13y ago
Currently using systemd on debian myself, with a few of my own custom .service files and a lot of falling back to sysvinit compatibility -- would love to see more upstream packages contain .service files. Things like "automatically restart if the service dies uncleanly" being handled by the init system are a godsend compared to managing init.d scripts which try to do that job themselves (things like "service start" getting into an infinite restart loop, then "service stop" says "service is not running", because the init script is stuck in a loop but the service itself hasn't been spawned...)
- peterwwillis 13y ago
  init scripts were originally designed to be "dumb" scripts that only performed one simple task. other scripts could then be written around them reliably, without wondering what the script might be doing in the background other than loading some options and running a program. with modern service management, a lot of extra "intelligence" has been added to how programs are executed and managed. this leads to levels of complexity and uncertainty that eventually lead administrators to hacking the hell out of the system to be able to use it reliably, usually disabling advanced features so they can control such features themselves. on top of that, automation of service restart is a solved problem since... decades ago? it's trivial to have a service restart if it's killed. unfortunately, once such a function is added to your init system, people use it all over the place and don't consider the consequences of services restarting automatically. eventually they get bitten by an unforeseen problem and build in limits to the service restart, etc etc. system automation is a lot harder than most people think.
  - bkor 13y ago
    A lot of Uncertainty and Doubt you're raising there... I'm wondering if you actually used systemd?
  - krakensden 13y ago
    > it's trivial to have a service restart if it's killed. unfortunately, once such a function is added to your init system, people use it all over the place and don't consider the consequences of services restarting automatically. eventually they get bitten by an unforeseen problem and build in limits to the service restart, etc etc. system automation is a lot harder than most people think. All the more reason to solve it once instead of in a billion different and differently awful addon packages. Plus, systemd is actually better at this than supervisord, daemontools, etc., already because of its cgroup usage- it can reliably kill all children before restarting, even if the process forks. Finally, I encourage you to read the systemd docs: http://www.freedesktop.org/software/systemd/man/systemd.service.html http://www.freedesktop.org/software/systemd/man/systemd.serv... they are quite nice, and show that, in fact, thought has gone into it. > unfortunately, once such a function is added to your init system, people use it all over the place I'm not super on board with this view- for almost everyone, if apache goes down, they want something to restart apache because the problem was probably transient and having apache being down is a Big Problem. Plus, most modern supervisors know about things like "restart throttling".
lamby 13y ago
This sort of thing actually happens a fair amount, albeit usually without a "celebrity" company and often without the organisation even being named. It's actually a difficult email to write (especially if you are disagreeing with a proposed change) as it can easily come across as a sort of childish blackmail.
- ersii 13y ago
  Well, to put some perspective into their "weighing in" on the issue: the guys at Spotify really do care about Free/Open Source Software. (Especially Noa, for what I've noticed). I interpret their "weighing in" on the issue as a balanced and constructive feedback claim to the community, from a community member. I think it's great when people participate and especially when they go full disclosure; iesaying who they are and from where and especially why (in a positive/constructive way).
- _delirium 13y ago
  Yeah, it seems like it could range from positive to negative depending on the precise tone and implications. On the one hand, if a large Debian user believes that some decisions will make lives easier for them and others harder, them explaining why they think so gives useful input to Debian that they can use to better server their users. Of course, one large company's interests might be outweighed by other considerations (and considerations of other users), but it's useful input to have either way. On the other hand, written in slightly the wrong way it can come across as bullying... "as one of your largest users, we strongly suggest you make this decision or we will have to pursue alternatives". Even more tricky in cases where the user is also a significant supporter of a project (whether through staff resources, donations, etc.). This one seemed reasonably written to me. It was a little light on the technical details of why specifically Spotify finds systemd works best for their use case, but the three bullet points do give some information.
deleted 13y ago
[deleted]
wpietri 13y ago
Two things I love about this: ops guys getting involved in the debate, and the fact that Spotify has somebody who's job is to be open-source ombudsman.
- dima55 13y ago
  The debate moved to the CTTE long ago. Stuff like this is noise at this point.
  - girvo 13y ago
    While you're right, don't be such a grump. It's cool that some business try and stay involved in this :)
possibilistic 13y ago
Debian.org's server seems to be having trouble. This is getting traffic from both HN and /r/linux. Cache: http://bugs.debian.org.nyud.net/cgi-bin/bugreport.cgi?msg=3546;bug=727708 http://bugs.debian.org.nyud.net/cgi-bin/bugreport.cgi?msg=35...
runjake 13y ago
I'm confused about debian stable vs testing vs unstable. Is there a branch that keeps relatively recent versions of packages and is reasonably usable as a daily driver?
- pilgrim689 13y ago
  People will recommend "testing" to you but I would advise against it... My path was: stable->testing->stable, because while on testing random stuff would break (laptop wouldn't sleep when I shut the lid, UI glitches, etc.). Plus, on testing I had to frequently update packages I didn't care about updating. I use stable now and when I need a recent package I now instead: compile from source/find backports/use a vm/vagrant It's the price I pay for a stable system. Vagrant for my dev environments makes "stable" a much easier choice.
- p4bl0 13y ago
  I use Debian on all my computers. The stable branch for servers and testing for others. Debian testing is sufficiently up-to-date for my taste. The very few things that are not can usually be installed anyway through backports or other repository. The last case I had (but that's a perhaps once in a year scenario) was with Batteries [1]. I needed version 2.1 but Debian testing only has 2.0, what I did is simply to install Batteries through opam [2] rather than apt-get install it. [1] http://batteries.forge.ocamlcore.org/ http://batteries.forge.ocamlcore.org/ [2] http://opam.ocaml.org/ http://opam.ocaml.org/
  - sandGorgon 13y ago
    do you ever use a PPA ? Most of the custom repos are meant for ubuntu and I cant ever find any which will respond to "sid" or "jessie" or something.
    - p4bl0 13y ago
      Nope I never had to use a PPA repository, sorry.
- networked 13y ago
zx2c4 13y ago
So that you don't kill their cgi bug tracker, here's the static archive: https://lists.debian.org/debian-ctte/2014/01/msg00287.html https://lists.debian.org/debian-ctte/2014/01/msg00287.html
_delirium 13y ago
Tangent, but the 2nd point is intriguing. How does one use cgroups to set up resource limitations? Is there any kind of decent front-end? I've seen the kernel documentation (https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt), but how do I use it? For example, if I wanted to give some shell users limited accounts where they can't use more than 512 RAM and some CPU quota each, I gather cgroups can do this, but I haven't been able to figure out how to set something like that up. (Yes, I know you could handle that use-case by giving each user their own virtual machine and let VMWare or Xen or Virtualbox handle the RAM/CPU quotas, but that's often not what I want.)
- WestCoastJustin 13y ago
  You're in luck -- I've done a screencast on cgroups where I explain what they are and how you can use them @ http://sysadmincasts.com/episodes/14-introduction-to-linux-control-groups-cgroups http://sysadmincasts.com/episodes/14-introduction-to-linux-c... Also, it is all command line at this point, but there are some concepts floating around about creating a gui @ http://mairin.wordpress.com/2011/05/13/ideas-for-a-cgroups-ui/ http://mairin.wordpress.com/2011/05/13/ideas-for-a-cgroups-u...
  - sandGorgon 13y ago
    could you double check your screencast ? My understanding is limited, but I think Systemd ver 205 changes the cgroup interaction methodology - basically systemd is a single point entry into the cgroup api. You cannot touch cgroup any other way except through systemd. FYI - this is not systemd taking arbitrary control but what the upstream kernel recommended. My understanding is that Fedora 20 (which uses systemd 208 ) will be impacted by this, as will most other distros. However jessie is proposing systemd 204, so your method might still work.
    - nisa 13y ago
      Is there a reason for this? I'm heavily invested in Ubuntu and I'm also making heavy use of the cgroups API in its current form. Does that mean that I can't use cgroups without systemd in newer kernels? Or do I miss cgroups features? I'm quite happy with the mountable filesystem at the moment. It's scriptable and reasonable flexible and still easy to use for solving my problems.
pekk 13y ago
It's a lot easier for one systemd-loving company to switch to RHEL/CentOS/Fedora than for everyone else to deal with problems with systemd. systemd is at least controversial enough in ways that matter to Debian (portability off Linux, etc.) that I think it's nice not to have a monoculture here.
- bkor 13y ago
  What systemd problems? Please elaborate.
- nailer 13y ago
  Really? I've been away from Linux for a few years, but it used to be my career. I recently checked out systemd to make a service on a CoreOS box. systemd replaces so much overlapping logic, has config files for services rather than scripts, and uses an existing known config format. Basically everything that sucked about SysV init. What's not to like? Edit: I see you've added portability off Linux. That's a very small concern, and certainly lower than 'deep Linux integration' for most Linux users.
  - pekk 13y ago
    Being better than SysV init is not something systemd has a monopoly on... Portability off Linux isn't a small concern for the Debian project as far as I know Red Hat related distros are very good distros based on systemd and are still available, it isn't necessary for Debian also to adopt systemd in order for people to use and enjoy it
    - Sanddancer 13y ago
      Debian has an active port to using the FreeBSD kernel, as well as a port to Hurd. So yes, portability off the Linux kernel is a concern to Debian.
  - justin66 13y ago
    > I see you've added portability off Linux. That's a very small concern, and certainly lower than 'deep Linux integration' for most Linux users. It didn't help systemd's popularity that the people behind it have tried to do disgusting stuff like convince GNOME to make GNOME dependent on it, which doesn't just effect "portability off linux" but the portability of GNOME. (I don't believe they got that one through)
agumonkey 13y ago
From linux conf australia 2014 : the 6 stages of systemd http://www.youtube.com/watch?v=-97qqUHwzGM‎ http://www.youtube.com/watch?v=-97qqUHwzGM‎
- rodgerd 13y ago
  The most remarkable thing about doing this talk was how many people have told me, either before (on hearing what I was talking about) or after, that they had various strong opinions on the topic, but hadn't spun up systemd on their servers. The most gratifying was the number of people who said they'd give it a try after the talk.
  - agumonkey 13y ago
    Was it based on previous software shipped by L.P (pulseaudio was a difficult exercise to say the least) ? or just because of change of decades old SystemV habits that felt heavy and potentially scary ?
    - rodgerd 13y ago
      I didn't conduct a scientific survey, but I think more the latter than the former; also, a lot of peoples' knowledge has been very much based on the public debate, which tends to the idea systemd offers benefits for desktops. Which may be true, but I see far more benefits for desktops.
      - rodgerd 13y ago
        (That should be "more benefit for servers", of course)
- jethro_tell 13y ago
  Yeah, that was me as well.
ForHackernews 13y ago
Who's responsible for changing the titles on these threads? You've removed the interesting bit: That it's Spotify's ops people putting in their two cents.
- thepumpkin1979 13y ago
  I submitted it, the title was changed and so was the author. What's going on here.
  - orik 13y ago
    Did they actually change the author of the post? I don't see it in your submission history anymore.
    - possibilistic 13y ago
      I submitted the post, but the title was changed from > Spotify responds to "Decide which init system to default to in Debian" I did mistakenly link to a dynamic resource, though.
  - ricardobeat 13y ago
    If you submit a duplicate URL, HN "accepts" it then redirects you to the earlier posting.
    - thepumpkin1979 13y ago
      That's probably what happened.
  - sp332 13y ago
    If you submit a link that someone else already posted, it just counts your "submission" as an upvote on the older post.
- tbarbugli 13y ago
  why on HN top op the page comments are always not about the content but about meta / secondary stuff ? ps. please dont be mean, do not upvote this!
forgottenpass 13y ago
Out of all the people commenting on #727708, The Spotify post is a rep from an organization with systems to administrate. Just like almost every other not-ctte, not-init maintainer person on that list, so it makes sense for Spotify to comment, but not really worth giving undue weight to their post in the larger debate currently happening on debian-ctte. If you want a better picture of debian's init decision, the bug got punted to ctte to decide on, so check out the ctte ml archives for December [0] and January [1]. Russ Allbery and Ian Jackson are the most vocal CTTE members on the list and support systemd and upstart respectively. The thread "Bug#727708: init system other points, and conclusion" is where Ian [2] Russ [3] start making the case for their preliminary conclusions. Edit: If there are other things worth pointing out on the list put 'em here. I've really only been skimming the posts for the last few weeks. They're also considering openRC, but it appears not to be a real contender. Tollef Fog Heen is a debian systemd maintainer and has posts worth reading on both what systemd actually is and what the impact of different decisions. [0] https://lists.debian.org/debian-ctte/2013/12/ https://lists.debian.org/debian-ctte/2013/12/ [1] https://lists.debian.org/debian-ctte/2014/01/ https://lists.debian.org/debian-ctte/2014/01/ [2] https://lists.debian.org/debian-ctte/2013/12/msg00182.html https://lists.debian.org/debian-ctte/2013/12/msg00182.html [3] https://lists.debian.org/debian-ctte/2013/12/msg00234.html https://lists.debian.org/debian-ctte/2013/12/msg00234.html
- rohansingh 13y ago
  I should point out that while this is theoretically the official position of our infrastructure team here at Spotify, we've also got a lot of opinion and debate internally (like you have on any topic among a large team of engineers). One other fun problem Spotify is going to tackle is how we're going to handle systemd vs. upstart if/when we transition from straight-up Debian to Ubuntu.
  - asdfaoeu 13y ago
    Why are you transitioning to Ubuntu?
    - rohansingh 13y ago
latchkey 13y ago
The bigger question here for me is why Spotify needs 5000 physical servers.
- brokenparser 13y ago
  Not only that, but why with that many servers pages (playlist top, artist/album details, etc) are often very slow to load.
Patrick_Devine 13y ago
I have to admit, I was dragged kicking and screaming into the new init world, and really didn't want to leave SysV init. I haven't used systemd, but I have been using upstart and it's grown on me. After reading this post, I did look at: https://wiki.debian.org/Debate/initsystem/upstart https://wiki.debian.org/Debate/initsystem/upstart Which does spell out some reasonable pros and cons of both systems. I'd say the biggest problem with upstart (and possibly the deal breaker) is the licensing terms. Yes, it's open source, but no, I really don't want to have to fill out Canonical's contributer license agreement if I want to contribute. I'm not really sure why Canonical doesn't just GPL like everyone else and leave it at that. There are some tools out there for upstart, but one thing which would be really nice would be a built in command for visualizing the init graph in ascii. It doesn't have to be fancy, just enough to know where your init script is going to be called in the boot process.
- hansjorg 13y ago
  > I'm not really sure why Canonical doesn't just GPL like everyone else and leave it at that That is about copyright, not about the license. Organizations who requires this includes the FSF and Mozilla for example. See: http://www.gnu.org/licenses/why-assign.html http://www.gnu.org/licenses/why-assign.html
  - bkor 13y ago
    But FSF/GNU contributions are guaranteed to stay under a Free Software license. Canonical is not similar. Not at all the same situation. Further, IMO FSF/GNU/Mozilla are also bad for requesting this.
  - warp 13y ago
    The FSF doesn't require this for all GNU projects. Each GNU project can decide if they want to require a CLA or not.
- andor 13y ago
  I'd say the biggest problem with upstart (and possibly the deal breaker) is the licensing terms. Upstart isn't even a real solution for Ubuntu: they still use SysV init scripts for many daemons (I don't know why exactly). Because upstart cannot monitor daemons that it started through init scripts, it's not reliable for process monitoring... Also see: https://wiki.debian.org/Debate/initsystem/systemd#Upstart https://wiki.debian.org/Debate/initsystem/systemd#Upstart
codingtheone 13y ago
I think this is the most important bit "dependency model of systemd is easier to understand, explain and work with than the event based counterpart of upstart.". IMO upstart is over-engineered to say the least.
colanderman 13y ago
Glad I'm not the only person who thinks upstart is back-assward. I've never understood how its model of "start a service iff all its dependencies are running" makes any sense: it (a) starts random other services I don't care about, just because their dependencies are met, and (b) forces me to manually track down and start all of a service's dependencies in order to start it. Though I'd be glad if someone could explain to me how it does make sense.
- CameronNemo 13y ago
  (a) makes sense because it is assumed that if you have the job enabled, you want it to be started. If you do not care about those services, just disable them. (b) is indeed a PITA.

234 points by possibilistic 13y ago

Just saw this interesting post by Noa at Spotify (speaking on his own behalf) later in the thread, about the trade-off between monoculture on the one extreme, and effort spread too thin between competing solutions on the other: https://lists.debian.org/debian-ctte/2014/01/msg00306.html https://lists.debian.org/debian-ctte/2014/01/msg00306.html

beefhash 13y ago

I don't understand how this is still anything to discuss. systemd is the only option; GNOME is already using it for some parts and will probably not hesitate to integrate it even more deeply to push Poettering's agenda. Whether or not the sysadmins want to learn systemd and regardless of whether it's the technically superior super-init system (you can't really call systemd an init system anymore), it's going to be forced down our throats anyway. What's with the holdup?

Nursie 13y ago
GNOME is far from the only FOSS desktop these days. A desktop that demands you have a specific init system is not onw I care to use.
bkor 13y ago
Systemd provides various things that desktop environments want to use. This is not limited to GNOME. I talked to Enlightment and they're going to make more use of various bits (e.g. user session). Similar with KDE. Dismissing it entirely like you're doing and pretending this is about forcing: good luck with that. Just like distributions were forced to use it right? Not merit, just pushed? Try coming up with something concrete. Until that the lack of specifics in your posts tells me enough.

dinkumthinkum 13y ago

The person that wrote is a "Free Software ombudsman" ... Pretty serious stuff we got going in software now. :)

trump 13y ago

Keep in mind, this is coming from the same company that gives their devs root: https://www.youtube.com/watch?v=pts6F00GFuU#t=169 https://www.youtube.com/watch?v=pts6F00GFuU#t=169 Otherwise, I agree with their endorsement of systemd.

bitwize 13y ago

Systemd is the Linux init system. Using anything else at this point is folly. This debate is a tempest in a teapot stirred up by whiners and haters who are resisting what the community overwhelmingly supports as a technically superior, easier, and designed-for-Linux init system.

akira2501 13y ago
It's better than SysV init, but there are a dozen other init's out there, all with different properties. Not everyone cares about "linux on the desktop," and not everyone thinks that freedesktop.org is doing the best job as far as that goes either.
- bitwize 13y ago
  It's better than SysV init, but there are a dozen other init's out there, all with different properties. And none of them leverage Linux's features as extensively as systemd. By the Linux init system, I mean the one overwhelmingly supported by most of the Linux community, and the one which is best designed to work with Linux. Not everyone cares about "linux on the desktop," and not everyone thinks that freedesktop.org is doing the best job as far as that goes either. Who's talking about desktops? Systemd is not a freedesktop project, and it was designed to make server administration easier by offering much finer grained control and monitoring of system services with a much better interface. Many of its most controversial features -- such as binary log files -- don't make sense except in a server context where the tampering-attestation features of systemd-journald make it a far more secure solution than plain text logs. You should really read Lennart's essays and blog posts about systemd to get a clearer picture of its tremendous advantages. Whatever you do, you'd best get used to systemd. It has all but won.
  - akira2501 13y ago
    I'm not trying to be snarky here; but systemd is hosted on freedesktop.org, is it not? That is its primary place of development as well, no? Admittedly, I've always associated systemd as a project as part of this nebulous "effort" to bring linux on the desktop forward. I also assumed that's why they went to the trouble to integrate systemd into dbus so heavily. I have read some of Lennart's essays -- but I generally strongly disagree with his conclusions, so I haven't been paying attention for a while. I think his solutions are over complicated and rely on too much impenetrable technology to actually save me any time. The arguments have been made elsewhere, so I won't go into it that deeply, but I think departure from the "unix way" is folly and will only make administration more difficult. As far as getting used to it goes; this is still unix. I can still install whatever initd I like after the fact. Which I do. Yes, this means I have to do more work than I would have to do if I relied on vendor packages, but we generally use custom packages for all our critical services anyways. Since we're going to have to deal with it whether or not we're using systemd, and we're already rolling out a non-standard initd to our distributions, we don't feel we have to make any investment in learning systemd.

myrandomcomment 13y ago

Both systemd and upstart have added a level of complexity I would prefer to avoid. The simplistically of sysvinit is greatly missed. But then again I am an old fuck so..

bkor 13y ago
The complexity is there, you just don't see it. Systemd handles all types of services. Sysvinit does not. Systemd abstracts a lot, making it easier per service (by putting more in systemd). In sysvinit a lot is copy/pasted across services. IMO making everything more error prone.

mmagin 13y ago

What made me hate upstart was that it still is willing to get the default runlevel from /etc/inittab, but it doesn't bother to emit warnings to the user that the rest of /etc/inittab is completely ignored, at least on the version of CentOS I was using. I think it's particularly user-hostile design for any program to accept configuration that does nothing (excepting whatever comment syntax a particular config file uses.) I would vastly prefer if it either made lots of complaints about the ignored lines in inittab, or even if it it failed hard -- because then I would have to fix the problem immediately. I have used systemd, but never actually poked around at its configuration (having just run Debian with it as a desktop user), so I can't properly judge it yet.

deleted 13y ago

[deleted]

quarterto 13y ago
Call me old school but I don't see why we can't just keep compiling everything from source. If one package has a dependency on another, just check if the other package is built or refuse to build. What is driving this need for "modern package management" ?
- keithpeter 13y ago
  Did a custom kernel under RHEL beta on this Thinkpad x61s core duo 2 laptop. Took 3 hours and used 16Gb of hard drive space. Compiling R takes about 20 minutes. Binaries are quicker.
- yogo 13y ago
  Because it's a time wasting process? Unless of course, everything you need to compile is relatively small.
  - _delirium 13y ago
    That's been my experience with MacPorts, which does have some binary packages, but much lower coverage than a distribution like Debian, so fallback building from source is common. It can be really annoying when you just want to install some utility and it takes literally 3 or 4 hours to install, because somewhere in the dependency or build-dependency chain is some big and slow-to-compile thing like GTK, X.org, or a newer GCC.
    - Zancarius 13y ago
      Gosh, you're reminding me of those days of compiling KDE or Firefox on an old box under Gentoo. I think I still wake up in a cold sweat from time to time. :)
  - Zancarius 13y ago
    Exactly. Time constraints and laziness are my two primary reasons for preferring a package manager over source compilation. That's not to say a good sysadmin shouldn't know how to compile from source--you should--but building every package from source and wasting time on a system with a strong package manager almost seems criminal or at least disrespectful to the people paying for that time. Package managers are a time saving device, and these days they're generally very good. If you have a need that isn't met by a modern package manager, I can't help but think it's an exceptionally niche use case. I used to compile many of my packages from source because of tunings that weren't present in the pre-built packages or pre-built packages that weren't available/out of date (this was probably on FreeBSD some 10+ years ago, although I'm reluctant to include building from ports in this because that is mostly automated and doesn't really count). But when I migrated to Gentoo where builds are automated (and then to Arch) and later experienced the Debian/Ubuntu side of things, I began to appreciate what a colossal waste of time it was to dig through configure options, set flags, etc., much less doing everything else manually. It's just more convenient to allow the platform itself to take care of things or offload worrying about dependencies to the package maintainer who probably knows more about building these packages than I do (or care to). Again, it's not to disparage the value of building from source (or knowing how), but I have other things I'd rather put my time into, like whining about compiling from source. ;)

cakeface 13y ago

I highly recommend that anyone interested in systemd who doesn't know much about it read Lennart Poettering's initial descriptions of it. I knew very little about Linux init systems and I found this incredibly interesting and informative. http://0pointer.de/blog/projects/systemd.html http://0pointer.de/blog/projects/systemd.html

pingswept 13y ago
Thanks for that. Very interesting read.

deleted 13y ago

[deleted]

Shish2k 13y ago

Currently using systemd on debian myself, with a few of my own custom .service files and a lot of falling back to sysvinit compatibility -- would love to see more upstream packages contain .service files. Things like "automatically restart if the service dies uncleanly" being handled by the init system are a godsend compared to managing init.d scripts which try to do that job themselves (things like "service start" getting into an infinite restart loop, then "service stop" says "service is not running", because the init script is stuck in a loop but the service itself hasn't been spawned...)

peterwwillis 13y ago
init scripts were originally designed to be "dumb" scripts that only performed one simple task. other scripts could then be written around them reliably, without wondering what the script might be doing in the background other than loading some options and running a program. with modern service management, a lot of extra "intelligence" has been added to how programs are executed and managed. this leads to levels of complexity and uncertainty that eventually lead administrators to hacking the hell out of the system to be able to use it reliably, usually disabling advanced features so they can control such features themselves. on top of that, automation of service restart is a solved problem since... decades ago? it's trivial to have a service restart if it's killed. unfortunately, once such a function is added to your init system, people use it all over the place and don't consider the consequences of services restarting automatically. eventually they get bitten by an unforeseen problem and build in limits to the service restart, etc etc. system automation is a lot harder than most people think.
- bkor 13y ago
  A lot of Uncertainty and Doubt you're raising there... I'm wondering if you actually used systemd?
- krakensden 13y ago
  > it's trivial to have a service restart if it's killed. unfortunately, once such a function is added to your init system, people use it all over the place and don't consider the consequences of services restarting automatically. eventually they get bitten by an unforeseen problem and build in limits to the service restart, etc etc. system automation is a lot harder than most people think. All the more reason to solve it once instead of in a billion different and differently awful addon packages. Plus, systemd is actually better at this than supervisord, daemontools, etc., already because of its cgroup usage- it can reliably kill all children before restarting, even if the process forks. Finally, I encourage you to read the systemd docs: http://www.freedesktop.org/software/systemd/man/systemd.service.html http://www.freedesktop.org/software/systemd/man/systemd.serv... they are quite nice, and show that, in fact, thought has gone into it. > unfortunately, once such a function is added to your init system, people use it all over the place I'm not super on board with this view- for almost everyone, if apache goes down, they want something to restart apache because the problem was probably transient and having apache being down is a Big Problem. Plus, most modern supervisors know about things like "restart throttling".

lamby 13y ago

This sort of thing actually happens a fair amount, albeit usually without a "celebrity" company and often without the organisation even being named. It's actually a difficult email to write (especially if you are disagreeing with a proposed change) as it can easily come across as a sort of childish blackmail.

ersii 13y ago
Well, to put some perspective into their "weighing in" on the issue: the guys at Spotify really do care about Free/Open Source Software. (Especially Noa, for what I've noticed). I interpret their "weighing in" on the issue as a balanced and constructive feedback claim to the community, from a community member. I think it's great when people participate and especially when they go full disclosure; iesaying who they are and from where and especially why (in a positive/constructive way).
_delirium 13y ago
Yeah, it seems like it could range from positive to negative depending on the precise tone and implications. On the one hand, if a large Debian user believes that some decisions will make lives easier for them and others harder, them explaining why they think so gives useful input to Debian that they can use to better server their users. Of course, one large company's interests might be outweighed by other considerations (and considerations of other users), but it's useful input to have either way. On the other hand, written in slightly the wrong way it can come across as bullying... "as one of your largest users, we strongly suggest you make this decision or we will have to pursue alternatives". Even more tricky in cases where the user is also a significant supporter of a project (whether through staff resources, donations, etc.). This one seemed reasonably written to me. It was a little light on the technical details of why specifically Spotify finds systemd works best for their use case, but the three bullet points do give some information.

deleted 13y ago

[deleted]

wpietri 13y ago

Two things I love about this: ops guys getting involved in the debate, and the fact that Spotify has somebody who's job is to be open-source ombudsman.

dima55 13y ago
The debate moved to the CTTE long ago. Stuff like this is noise at this point.
- girvo 13y ago
  While you're right, don't be such a grump. It's cool that some business try and stay involved in this :)

possibilistic 13y ago

Debian.org's server seems to be having trouble. This is getting traffic from both HN and /r/linux. Cache: http://bugs.debian.org.nyud.net/cgi-bin/bugreport.cgi?msg=3546;bug=727708 http://bugs.debian.org.nyud.net/cgi-bin/bugreport.cgi?msg=35...

runjake 13y ago

I'm confused about debian stable vs testing vs unstable. Is there a branch that keeps relatively recent versions of packages and is reasonably usable as a daily driver?

pilgrim689 13y ago
People will recommend "testing" to you but I would advise against it... My path was: stable->testing->stable, because while on testing random stuff would break (laptop wouldn't sleep when I shut the lid, UI glitches, etc.). Plus, on testing I had to frequently update packages I didn't care about updating. I use stable now and when I need a recent package I now instead: compile from source/find backports/use a vm/vagrant It's the price I pay for a stable system. Vagrant for my dev environments makes "stable" a much easier choice.
p4bl0 13y ago
I use Debian on all my computers. The stable branch for servers and testing for others. Debian testing is sufficiently up-to-date for my taste. The very few things that are not can usually be installed anyway through backports or other repository. The last case I had (but that's a perhaps once in a year scenario) was with Batteries [1]. I needed version 2.1 but Debian testing only has 2.0, what I did is simply to install Batteries through opam [2] rather than apt-get install it. [1] http://batteries.forge.ocamlcore.org/ http://batteries.forge.ocamlcore.org/ [2] http://opam.ocaml.org/ http://opam.ocaml.org/
- sandGorgon 13y ago
  do you ever use a PPA ? Most of the custom repos are meant for ubuntu and I cant ever find any which will respond to "sid" or "jessie" or something.
  - p4bl0 13y ago
    Nope I never had to use a PPA repository, sorry.
networked 13y ago

zx2c4 13y ago

So that you don't kill their cgi bug tracker, here's the static archive: https://lists.debian.org/debian-ctte/2014/01/msg00287.html https://lists.debian.org/debian-ctte/2014/01/msg00287.html

_delirium 13y ago

Tangent, but the 2nd point is intriguing. How does one use cgroups to set up resource limitations? Is there any kind of decent front-end? I've seen the kernel documentation (https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt), but how do I use it? For example, if I wanted to give some shell users limited accounts where they can't use more than 512 RAM and some CPU quota each, I gather cgroups can do this, but I haven't been able to figure out how to set something like that up. (Yes, I know you could handle that use-case by giving each user their own virtual machine and let VMWare or Xen or Virtualbox handle the RAM/CPU quotas, but that's often not what I want.)

WestCoastJustin 13y ago
You're in luck -- I've done a screencast on cgroups where I explain what they are and how you can use them @ http://sysadmincasts.com/episodes/14-introduction-to-linux-control-groups-cgroups http://sysadmincasts.com/episodes/14-introduction-to-linux-c... Also, it is all command line at this point, but there are some concepts floating around about creating a gui @ http://mairin.wordpress.com/2011/05/13/ideas-for-a-cgroups-ui/ http://mairin.wordpress.com/2011/05/13/ideas-for-a-cgroups-u...
- sandGorgon 13y ago
  could you double check your screencast ? My understanding is limited, but I think Systemd ver 205 changes the cgroup interaction methodology - basically systemd is a single point entry into the cgroup api. You cannot touch cgroup any other way except through systemd. FYI - this is not systemd taking arbitrary control but what the upstream kernel recommended. My understanding is that Fedora 20 (which uses systemd 208 ) will be impacted by this, as will most other distros. However jessie is proposing systemd 204, so your method might still work.
  - nisa 13y ago
    Is there a reason for this? I'm heavily invested in Ubuntu and I'm also making heavy use of the cgroups API in its current form. Does that mean that I can't use cgroups without systemd in newer kernels? Or do I miss cgroups features? I'm quite happy with the mountable filesystem at the moment. It's scriptable and reasonable flexible and still easy to use for solving my problems.

pekk 13y ago

It's a lot easier for one systemd-loving company to switch to RHEL/CentOS/Fedora than for everyone else to deal with problems with systemd. systemd is at least controversial enough in ways that matter to Debian (portability off Linux, etc.) that I think it's nice not to have a monoculture here.

bkor 13y ago
What systemd problems? Please elaborate.
nailer 13y ago
Really? I've been away from Linux for a few years, but it used to be my career. I recently checked out systemd to make a service on a CoreOS box. systemd replaces so much overlapping logic, has config files for services rather than scripts, and uses an existing known config format. Basically everything that sucked about SysV init. What's not to like? Edit: I see you've added portability off Linux. That's a very small concern, and certainly lower than 'deep Linux integration' for most Linux users.
- pekk 13y ago
  Being better than SysV init is not something systemd has a monopoly on... Portability off Linux isn't a small concern for the Debian project as far as I know Red Hat related distros are very good distros based on systemd and are still available, it isn't necessary for Debian also to adopt systemd in order for people to use and enjoy it
  - Sanddancer 13y ago
    Debian has an active port to using the FreeBSD kernel, as well as a port to Hurd. So yes, portability off the Linux kernel is a concern to Debian.
- justin66 13y ago
  > I see you've added portability off Linux. That's a very small concern, and certainly lower than 'deep Linux integration' for most Linux users. It didn't help systemd's popularity that the people behind it have tried to do disgusting stuff like convince GNOME to make GNOME dependent on it, which doesn't just effect "portability off linux" but the portability of GNOME. (I don't believe they got that one through)

agumonkey 13y ago

From linux conf australia 2014 : the 6 stages of systemd http://www.youtube.com/watch?v=-97qqUHwzGM‎ http://www.youtube.com/watch?v=-97qqUHwzGM‎

rodgerd 13y ago
The most remarkable thing about doing this talk was how many people have told me, either before (on hearing what I was talking about) or after, that they had various strong opinions on the topic, but hadn't spun up systemd on their servers. The most gratifying was the number of people who said they'd give it a try after the talk.
- agumonkey 13y ago
  Was it based on previous software shipped by L.P (pulseaudio was a difficult exercise to say the least) ? or just because of change of decades old SystemV habits that felt heavy and potentially scary ?
  - rodgerd 13y ago
    I didn't conduct a scientific survey, but I think more the latter than the former; also, a lot of peoples' knowledge has been very much based on the public debate, which tends to the idea systemd offers benefits for desktops. Which may be true, but I see far more benefits for desktops.
    - rodgerd 13y ago
      (That should be "more benefit for servers", of course)
jethro_tell 13y ago
Yeah, that was me as well.

ForHackernews 13y ago

Who's responsible for changing the titles on these threads? You've removed the interesting bit: That it's Spotify's ops people putting in their two cents.

thepumpkin1979 13y ago
I submitted it, the title was changed and so was the author. What's going on here.
- orik 13y ago
  Did they actually change the author of the post? I don't see it in your submission history anymore.
  - possibilistic 13y ago
    I submitted the post, but the title was changed from > Spotify responds to "Decide which init system to default to in Debian" I did mistakenly link to a dynamic resource, though.
- ricardobeat 13y ago
  If you submit a duplicate URL, HN "accepts" it then redirects you to the earlier posting.
  - thepumpkin1979 13y ago
    That's probably what happened.
- sp332 13y ago
  If you submit a link that someone else already posted, it just counts your "submission" as an upvote on the older post.
tbarbugli 13y ago
why on HN top op the page comments are always not about the content but about meta / secondary stuff ? ps. please dont be mean, do not upvote this!

forgottenpass 13y ago

Out of all the people commenting on #727708, The Spotify post is a rep from an organization with systems to administrate. Just like almost every other not-ctte, not-init maintainer person on that list, so it makes sense for Spotify to comment, but not really worth giving undue weight to their post in the larger debate currently happening on debian-ctte. If you want a better picture of debian's init decision, the bug got punted to ctte to decide on, so check out the ctte ml archives for December [0] and January [1]. Russ Allbery and Ian Jackson are the most vocal CTTE members on the list and support systemd and upstart respectively. The thread "Bug#727708: init system other points, and conclusion" is where Ian [2] Russ [3] start making the case for their preliminary conclusions. Edit: If there are other things worth pointing out on the list put 'em here. I've really only been skimming the posts for the last few weeks. They're also considering openRC, but it appears not to be a real contender. Tollef Fog Heen is a debian systemd maintainer and has posts worth reading on both what systemd actually is and what the impact of different decisions. [0] https://lists.debian.org/debian-ctte/2013/12/ https://lists.debian.org/debian-ctte/2013/12/ [1] https://lists.debian.org/debian-ctte/2014/01/ https://lists.debian.org/debian-ctte/2014/01/ [2] https://lists.debian.org/debian-ctte/2013/12/msg00182.html https://lists.debian.org/debian-ctte/2013/12/msg00182.html [3] https://lists.debian.org/debian-ctte/2013/12/msg00234.html https://lists.debian.org/debian-ctte/2013/12/msg00234.html

rohansingh 13y ago
I should point out that while this is theoretically the official position of our infrastructure team here at Spotify, we've also got a lot of opinion and debate internally (like you have on any topic among a large team of engineers). One other fun problem Spotify is going to tackle is how we're going to handle systemd vs. upstart if/when we transition from straight-up Debian to Ubuntu.
- asdfaoeu 13y ago
  Why are you transitioning to Ubuntu?
  - rohansingh 13y ago

latchkey 13y ago

The bigger question here for me is why Spotify needs 5000 physical servers.

brokenparser 13y ago
Not only that, but why with that many servers pages (playlist top, artist/album details, etc) are often very slow to load.

Patrick_Devine 13y ago

I have to admit, I was dragged kicking and screaming into the new init world, and really didn't want to leave SysV init. I haven't used systemd, but I have been using upstart and it's grown on me. After reading this post, I did look at: https://wiki.debian.org/Debate/initsystem/upstart https://wiki.debian.org/Debate/initsystem/upstart Which does spell out some reasonable pros and cons of both systems. I'd say the biggest problem with upstart (and possibly the deal breaker) is the licensing terms. Yes, it's open source, but no, I really don't want to have to fill out Canonical's contributer license agreement if I want to contribute. I'm not really sure why Canonical doesn't just GPL like everyone else and leave it at that. There are some tools out there for upstart, but one thing which would be really nice would be a built in command for visualizing the init graph in ascii. It doesn't have to be fancy, just enough to know where your init script is going to be called in the boot process.

hansjorg 13y ago
> I'm not really sure why Canonical doesn't just GPL like everyone else and leave it at that That is about copyright, not about the license. Organizations who requires this includes the FSF and Mozilla for example. See: http://www.gnu.org/licenses/why-assign.html http://www.gnu.org/licenses/why-assign.html
- bkor 13y ago
  But FSF/GNU contributions are guaranteed to stay under a Free Software license. Canonical is not similar. Not at all the same situation. Further, IMO FSF/GNU/Mozilla are also bad for requesting this.
- warp 13y ago
  The FSF doesn't require this for all GNU projects. Each GNU project can decide if they want to require a CLA or not.
andor 13y ago
I'd say the biggest problem with upstart (and possibly the deal breaker) is the licensing terms. Upstart isn't even a real solution for Ubuntu: they still use SysV init scripts for many daemons (I don't know why exactly). Because upstart cannot monitor daemons that it started through init scripts, it's not reliable for process monitoring... Also see: https://wiki.debian.org/Debate/initsystem/systemd#Upstart https://wiki.debian.org/Debate/initsystem/systemd#Upstart

codingtheone 13y ago

I think this is the most important bit "dependency model of systemd is easier to understand, explain and work with than the event based counterpart of upstart.". IMO upstart is over-engineered to say the least.

colanderman 13y ago

Glad I'm not the only person who thinks upstart is back-assward. I've never understood how its model of "start a service iff all its dependencies are running" makes any sense: it (a) starts random other services I don't care about, just because their dependencies are met, and (b) forces me to manually track down and start all of a service's dependencies in order to start it. Though I'd be glad if someone could explain to me how it does make sense.

CameronNemo 13y ago
(a) makes sense because it is assumed that if you have the job enabled, you want it to be started. If you do not care about those services, just disable them. (b) is indeed a PITA.