9 ms·
Dear Package Maintainers: Stop Being Clever
- stonith 13y agoWe have a similar problem when deploying Openstack using puppet - all the services need to be brought up after config that sets things like the central DB and AMQP locations. It means that a deployed site will always have errors in their logs right from the start, which confuses customers and makes it harder to spot when there is actually an error in the deployment tool.
- Patrick_Devine 13y agoIf you're using Ubuntu, the .deb packages need a lot of work. I've been playing around with this for the past week and there are all sorts of little glitches and gotchas. A big problem is that the package maintainers didn't use debconf, so you end up having to install, set some settings and then bring the service up. This is definitely not the debian/ubuntu way of doing things. If I find some time I'll see if I can repackage things to properly use debconf and push things upstream.
- Daviey 13y agoNo, Ubuntu specifically made the choice to avoid debconf for Openstack packages. If you are using debconf interactively, it simply doesn't scale - if you are preseeding - or using juju, puppet or chef - what value does debconf add? Please don't assume that it is simply wrong, just because it doesn't do x or y. The debconf templates will always be trying to play catchup, and the transition of options between components doesn't make a lot of sense. It would be really good to know what issues you have identified, as we take quality very seriously in this area. However, your help with packaging will be most appreciated. Feel free to give me a shout on IRC, and we'll go through it.
- Patrick_Devine 13y agoI'm not quite sure I'm parsing your sentence correctly. If you're preseeding with debconf the value is that you got to preseed the correct configuration options. This means the debs will work with anything, not just the flavour of the week devops tool. Juju, Puppet and Chef aren't the only games in town. I'll agree that they are all very useful tools, but you shouldn't need to write complex scripts/manifests/whatever around them to stand up services. That should happen out of the box without requiring anyone to jump through 50 hoops. It also means that you can scale from 1 instance all the way up to 5000 instances which is a lot better than optimizing for the 5000 instance case. I'll definitely give you a shout on IRC.
- Qantourisc 13y agoI agree on the autostart.
- codezero 13y agoI remember installing Red Hat 4, going on IRC and getting rooted. Red Hat's approach comes from lots of learning, and I think it's smart. Thanks for the post.
- antihero 13y agoCould you elaborate as to how this happened?
- codezero 13y agoLots of not up to date services were installed and activated by default. Exploits were abundant.
- jlgaddis 13y agoYears ago, Linux distributions defaulted to installing everything and the kitchen sink and enabling all those services automatically. After years of "OOTB pwnage", they started to make things a bit more secure by default and they eventually got to the point we're at today (where most installations are relatively secure by default, OOTB). (Side note: I believe it was RHL 5.2 that shipped with wu-ftpd enabled out of the box. By the time it shipped (or shortly afterwards, I forget which), there was a remote root exploit for wu-ftpd. Thus, if you installed RHL 5.2, you'd likely get pwned in short order.)
- codezero 13y agoThis is exactly correct. I was a Slackware zealot for a long time but frequently tried Red Hat because my boss back then was a big fan. I didn't actually convert to appreciating Red Hat until I worked there and learned the advantages of using RPM and other tools included in Red Hat that were absent in Slackware. In either scenario, and this is still true today, if you plan to connect your system to the Internet, you need to know your configuration and care for the system after it is live. Modern distros help with this a lot by offering rapid package updates, automatic updating and smarter firewall and security measures. A lot of this stuff was controlled by very esoteric text configuration files in the past, which made it hard even for a savvy person to get right every time.
- belorn 13y agoWhen packaging a daemon for Debian, the maintainer should add sufficient questions during installation, and sane default values, to start the service. Contrary to the article, I think this is a rather good policy. It makes sure packages are usable as they are, rather than the normal process of config, make, make-install, and last adding user configuring before anything will start up. Debian is not Gentoo. For those daemons which this isn't possible, package maintainers has a harder time. One of the better ways is simply to have it run, but in a safe way which do nothing. That would solve the issues the article describes, and would not be harmful.
- stephen_g 13y agoWhat does packages using non-standard hacks that break puppet scripting have to do with configuring and building source code? That's not related to the issue at all...
- belorn 13y agoThe article talks about two issues. One is that debian has policy to default run daemons after installation. The second issue the article brought it up is the issues with puppet. Regarding the first issue, sane defaults and usable packages directly after installation came as a result of the config, make, and make-install process. Users should get a useful package directly without needing to spend days adding all the default settings to start a simple mail or web daemon. The second issue has nothing to do with config, make, and make-install. It has to do with non-standard hacks. If nagios3 had simply started up like apache, this would have been a non-issue.
- _pmf_ 13y ago> One of the better ways is simply to have it run, but in a safe way which do nothing. That would solve the issues the article describes, and would not be harmful. I'd argue it would be worse. I'd rather have the program not run at all than have it preconfigured in a way that contradicts its regular operation and having to reconfigure it to make it work normally.
- 13y ago
- bashtoni 13y agoI posted this as a comment on the blog, but I'm duplicating here: RHEL and Debian (and their respective derivatives) have different philosophies on packaging, which is why you see this result. The RHEL philosophy is simple; package installation only installs the package, starting daemons is a separate process. The Debian philosophy is that installing a daemon should also make that daemon run, unless it's not possible the make the daemon run securely by default. Things like the /etc/default RUN="no" aren't the package maintainer trying to be clever, but rather them following Debian packaging guidelines. There also the reason you see things like MySQL listening only on the loopback interface by default on Debian. One you understand this the decisions seem a bit less crazy. For me, the big issue with the Debian approach isn't on installation, but rather on upgrade. On Debian, running 'apt-get upgrade' on a live server can cause an unspecified amount of downtime. If you're upgrading MySQL, MySQL will get restarted. If you're upgrading a library on which multiple daemons rely, they will also get restarted. yum upgrade on RHEL doesn't suffer from the same problem, and you can schedule the restart yourself. Again, this is because of a philosophical difference. On Debian, the idea is that if you've asked to upgrade something, it should be upgraded. On RHEL, you can upgrade a package, but continue to run the old (potentially insecure version).
- Shish2k 13y ago> If you're upgrading MySQL, MySQL will get restarted. Or specifically, it will get stopped, your packages will spend 15 minutes updating (or longer if there's an interactive prompt mid way and you aren't paying attention), and then it will get started again. I'm quite happy with the debian way of doing things, but I'd really like it if apt-get was smart enough to stop / upgrade / start individual services, rather than stop-all / upgrade-all / start-all...
- viraptor 13y ago15 minutes? Mysql upgrade and restart takes up to 5 second in a usual case. Also services do get stopped / started one by one, not all at once.
- 13y ago
- dmourati 13y agoI answered the serverfault post. http://serverfault.com/a/526123/76707 http://serverfault.com/a/526123/76707 I'm a longtime Redhat/CentOS guy recently moved over to Ubuntu. It has definitely been a bit weird getting adjusted. One counterpoint to the OP's rant? SSH.
- jmomo 13y agoI hate to say it, but this is just a rant by a RedHat/CentOS admin who has started using Debian/Ubuntu for the first time and just doesn't know why things are done differently there.
- jrochkind1 13y agodo you really hate to say it?
- MPSimmons 13y agoAuthor here. You're 99% correct. I'm historically an RHEL-based admin, who has started using Debian/Ubuntu for the first time (relatively, for about a year), but I totally get why these things are done. What I deeply disagree with is how they're done. Why are there so many non-standard ways that a package can decide to not start? I really appreciate that the package maintainer is trying to do the right thing, but it's ridiculous that there isn't a way to do it - there's the Debian default and then there's whatever the packager thought up. That's nuts, and needing to manage the same resource twice in puppet isn't a tragedy or big deal, but it's a symptom that something is wrong.
- sciurus 13y agoYou keep saying things like "so many non-standard ways that a package can decide to not start" and "whatever the packager thought up". Can you provide examples of what you mean? In your blog post you only mention one way of controlling startup- via an environment variable sourced from a file in /etc/defaults. This is the standardized, documented place in Debian for controlling the behavior of init scripts. Offhand, I can't think of any other ways I've seen packager maintainers controlling startup. I'd really need to see more evidence before agreeing that things are "ridiculous", "nuts", or "wrong". http://www.debian.org/doc/manuals/maint-guide/dother.en.html#initd http://www.debian.org/doc/manuals/maint-guide/dother.en.html...
- jsight 13y agoThe file is a standard, but the contents are not. For the most part, that makes sense, but it would be nice if there were a few standards there. Eg, DISABLE_ALL_SERVICES=true|false. :) More examples here: http://serverfault.com/questions/526075/fixing-services-that-have-been-disabled-in-etc-default-with-puppet http://serverfault.com/questions/526075/fixing-services-that... As it is, figuring out exactly how to do this has to be done for each service. It's tedious, IMO, though I understand the history of why this is the case.
- txutxu 13y agoWhat I don't understand is why every maintainer adds a (unneeded) layer for enable/disable. If you want to run a service, you have a runlevel editor. This is, chkconfig in RH based, and update-rc.d in debian based. That is the tool for the job. Nowadays it seems that everybody prefers to invent an additional switch, this is a shell boolean or string, in /etc/sysconfig (RH) or /etc/default (debian). Why ?
- moe 13y agoThe author is spot on. This has been a long-standing bug in Debian which, due to their glacial processes, will probably prevail eternally. In theory it is possible to disable the post installation auto-start with a specially crafted 'policy-rc.d'. In practice you don't want to go there, as it tends to break in unexpected ways.
- reidrac 13y agoEvery package maintainer needs to make some decisions when packaging software. Debian has very specific guidelines about packing, so if you don't like it may be Debian is the wrong distribution _for you_. Recently a workmate installed nginx from the upstream repo and... surprise: nginx packages things different to Debian. You may like it more or less, but there are different solutions for the same problem.
- MPSimmons 13y agoAuthor of the article here. Thanks, and I agree with you. I have some disagreements with the way Ubuntu does things like this, and I'd prefer an RHEL clone (which, of course, has tradeoffs too). That being said, I'm in the environment relatively recently and it's established, and I'm the one who has to learn to adapt since I don't see our infrastructure changing dramatically soon. Other than finally adding configuration management, that is.
- rlpb 13y agoDebian's thinking on this is that a user won't install a daemon unless he wants it to run, and that a package should get everything into an immediately useable state by default. Therefore, a daemon should be started automatically when a package is installed. I agree with the objection: I think that times have moved on, and in reality today people use configuration management tools and almost always want to change the default configuration on a server daemon. Therefore, it doesn't make much sense to start it automatically. As bashtoni says: this isn't a problem with package maintainers. It's a problem with Debian Policy. A stepping stone to a full solution is to arrange a standard to configure the system not to start a daemon by default, and arrange for packages to follow that standard. policy-rc.d is just such a standard.
- lmm 13y agoDebian always think they know better than the authors of the software they're distributing (the worst, but by no means only incident being their key generation bug). When I maintained a linux program I received twice as many bugs from Debian users as anything else, because Debian repackaged KDE in a way that broke its autoconf macros. And anyone who's ever tried to follow a tutorial for e.g. Tomcat will know how radically Debian rearranges its files. IMO it's not worth maintaining Debian packages of one's software or supporting it on Debian. Let Debian have their own packager for it if they want to, and direct support requests to them.
- Daviey 13y agoPerhaps you were seeing more bug reports from Debian users, because more people were using your software via Debian than elsewhere?
- lmm 13y agoRight, but the excess was all people with the same problem. I added a note to the homepage ("debian users: you need to install libblah-devel, configure will not warn you about this because of how debian packages KDE") but it didn't seem to make any difference.
- Chris2048 13y agoI believe ElasticSearch will automatically connect to any nodes with the same cluster-name. The package installs it with the default cluster-name. Since it starts on install, you don't have a chance to change this name before it's done that...
- dehora 13y agoApache's Cassandra deb that a similar behaviour and has a greater impact than ES's super-hero node names. A newly started Cassandra node will absent a token declaration in its yaml file, auto-assign a token. This is never what you want in a cluster distribution. So the workaround is either to patch the deb to not autostart or let it start, stop it, and blow away the data directories. This allows cluster management tools (such as Priam) to control the token allocation first, and then re-start Cassandra. This is effectively how Debian/Ubuntu work. Once you know it, you can work around it. That said, it would be nice to have packages that don't run their service.
- GrinningFool 13y agoIt seems like a lot of these comments are missing the point by discussing virtues of various packaging systems and auto-start behaviors. It seems to me that the author was saying that consistency is important - installed services are expected to have a consistent behavior on a given platform. A lot of tooling is built based on this, because that behavior is defined by the respective systems. When people force override of these behaviors, it does break things as well as cause frustration. The article isn't a criticism of any one packaging system or autostart choice of a distribution, but rather of individual packagers who impose their own choices over those of the system they're packaging for.