12 ms·

I still can't understand why several BSDs so stubbornly cling to ports as a "solution" to package management. I ran into this problem, and variants of it, over

I still can't understand why several BSDs so stubbornly cling to ports as a "solution" to package management. I ran into this problem, and variants of it, over and over again when trying to support FreeBSD in our products, in the past. Though pkgng may resolve some of my problems with package management on FreeBSD (it at least allows third party repositories, the lack of which was a big pain point for me in the past), I haven't tried it yet, and from what I can tell FreeBSD still favors ports, and several other BSDs are still using ports exclusively. I've even seen this library mismatch happen on a couple of occasions in a fresh install, where installing a variety of ports leads to one or some of them being broken immediately after install because several packages called for different versions of libraries. And, the longer a BSD system is in service, the more likely some portupgrade, or installation of a new package, is to break some other thing on the system, requiring far reaching rebuilds (and sometimes, manual intervention, because the dependencies specified aren't sufficient to allow the system to automatically fix itself). I begin to feel like a broken record saying this again and again, but: ports is a clever tool, but it is not a solution to the problem of package management. There's so much to love about the BSDs, but lack of effective package management has always made it a non-starter for any purpose, for me. Effective package management is not esoteric or bleeding edge or experimental stuff, at this point. deb-based systems had apt 20+ years ago, RPM based systems had yum ~12 years ago. Slapping a band aid on the darned thing doesn't fix it. And, so many conversations about packages devolve into ports proponents missing the point, and saying, "Well, you can install binary versions of ports, so it's fine." When the availability of binary packages is utterly orthogonal to the problems I'm talking about (though also a big deal, and building from source on a production system is silly and potentially dangerous). So, I think I need to stop complaining about it, and start asking the right questions of BSD users. Why is ports such a fiercely defended thing when it has so many weaknesses compared to modern package managers? Does pkgng actually solve these problems (I can't really tell, it seems to be a meta-package system that still sits atop ports)?

throwaway2048 11y ago
Should be noted that OpenBSD has package tools and management much more along the lines of apt or yum. Ports on openbsd build into packages so you dont get a weirdly broken system if you do need to dip into ports for some reason (although the general recommendation is to use binary packages)
- protomyth 11y ago
  I don't think I've used ports on FreeBSD and have stuck with pkg. PC-BSD has a pretty extensive package management system[1], and and NetBSD's package management is used by multiple operating systems[2]. Ports isn't really used unless you have some pretty specific needs. 1) http://web.pcbsd.org/doc-archive/10.1.2/html/update.html http://web.pcbsd.org/doc-archive/10.1.2/html/update.html 2) http://www.netbsd.org/docs/software/packages.html http://www.netbsd.org/docs/software/packages.html
vezzy-fnord 11y ago
Library mismatches, dependency conflicts and progressive destabilization/entropy (system package configurations approaching global failure conditions like boot stalling as time passes and upgrades are made) were all frequent experiences I had on dpkg-based systems with the APT frontend that ultimately made me return to Slackware and its spartan package management where I have since experienced nothing of the sort. This is likely a YMMV issue.
marios 11y ago
OpenBSD has both binary packages and ports. Actually, the packages are built from ports, so if you want to customize a package, you checkout the ports tree, adjust the Makefile, add patches or whatever it is you need, and then you get a nice package that you can use on other OpenBSD installations (provided they are running the same version). This is similar to what one does with, say, APT (apt-get {build-dep, source} <pkg>; modify; dpkg-builpackage ...). When you get install a release, you set a package source (/etc/pkg.conf is more or less the equivalent of /etc/apt/*), and then you just pkg_add <pkg>. Package versions are fixed, pretty much the same way you get when you run debian stable. There is one major difference though: OpenBSD is an OS, while Debian is a distribution. This means that debian will install the package files relatively to /, while OpenBSD will install them relatively to /usr/local. It may sound stupid if you've only used Linux before, but it makes sense: files in /usr/bin, /usr/sbin,... are files that were shipped with the OS. What lives in /usr/local is what you installed on top of it. Other than that, I don't see any major difference. You occasionally get some libraries that are out of sync with a package, but that happens on -current and it's usually that the package was not rebuilt yet on the mirror you're using. (Ever run Debian sid ? Well, it's the same thing) What I'm saying is that OpenBSD uses ports effectively and in day to day usage I see no difference to APT-based distributions. I've never used it, but I hear very good things about pkgsrc (developed by NetBSD but it can also be used elsewhere. AFAIK, Joyent uses it and some people run it on Linux too). From what I can see on the homepage, it provides both sources and binaries. Until pkgng was introduced, FreeBSD was lagging behind but from what I can tell, it seems the situation has improved. What "weaknesses" related to ports do you have in mind ?
- SwellJoe 11y ago
  "What "weaknesses" related to ports do you have in mind ?" The ones I've mentioned (and mentioned in the article we're commenting on): - Library mismatches leading to broken packages, sometimes on a freshly installed system. I've always heard outright denial that this happens, but I've literally never installed a FreeBSD system that didn't end up with something broken by the time everything needed for a fully functioning web hosting system was installed. It's like there's Stockholm syndrome happening, or something. - Poor dependency resolution, and weak validation tools, that requires manual intervention to prevent or repair those breakages which are bound to occur. Other complaints I have about ports: - No reasonable mechanism for offering third party package repositories. - Default ports are usually anemic, missing very basic functionality, guaranteeing you have to build some things from source in order to get a really functional version of, say, Apache. Now, you have to make your own means of distributing these custom builds across your systems, without any reasonable means of setting up your own repository (you can duplicate the whole ports tree, and merge your changes, which is ridiculous, and the tools to help with this are weak). Really, the biggest failings are in dependency specification and resolution. I don't know if it is a process failure (e.g., nobody is taking the time to specify things precisely enough to end up with a working package in every circumstance) or failure of the system (e.g., the system doesn't provide the means to specify things precisely enough to insure a functional system), or a combination of the two. But, the end result is that it is frightfully easy to end up with broken software, and often requires a lot of knowledge and human interaction to fix it. Again, providing binaries is fine and all (and, I think, mandatory for large automated deployments), but is not the basis of the problems I've experienced with ports. The fragility of ports is my problem with it, not the surface level differences between how things are installed and updated.
2trill2spill 11y ago
> So, I think I need to stop complaining about it, and start asking the right questions of BSD users. Why is ports such a fiercely defended thing when it has so many weaknesses compared to modern package managers? First theres no dichotomy here. A person can use ports and pkgng. Secondly there's many nice thing about ports. If your organization has custom patches or compile options for common software, you can use the port system. Also I think a more important use of ports, is security fixes. When OpenSSL drops it's bug of the week you can patch openssl and rebuild the port tree. Then you can push the new, patched binaries into production. A sysadmin can do this all without relying on the package maintainer, ie no waiting on Redhat to push out new rpms. This can all be done in under an hour depending on the performance of the build server and automation processes in place. > Does pkgng actually solve these problems (I can't really tell, it seems to be a meta-package system that still sits atop ports)? Yes. > and building from source on a production system is silly and potentially dangerous Well of course you don't build directly on the production box. That's what a build server is for. You can use poudriere to build packages from the ports tree and then run your own package repo and have your production boxes pull from your private package repo.
- SwellJoe 11y ago
  "Also I think a more important use of ports, is security fixes. When OpenSSL drops it's bug of the week you can patch openssl and rebuild the port tree. Then you can push the new, patched binaries into production. A sysadmin can do this all without relying on the package maintainer, ie no waiting on Redhat to push out new rpms." I can't remember the last time I was aware of a security problem before Red Hat (and CentOS) had rolled out a fixed package. I usually know about security issues when I see a slew of update notifications from my servers (even though I'm on several security related mailing lists). Realistically, I rely on my OS vendor to provide my security fixes in a timely manner. If my systems security relied on me being able to apply patches manually in a timely manner, it would be a disaster waiting to happen. And, it is a big pile of labor I don't want on my plate. Besides, nothing stops one from installing the source package (whether RPM or deb), adding the patch, rebuilding, and pushing to your local yum or apt repository. That workflow is entirely reasonable on Linux, as well (historically easier, since maintaining a third party repo was so much easier for yum, though apt is a bit more cantankerous; it looks like FreeBSD finally supports multiple repositories and you don't need to mirror the whole ports tree just to provide a handful of custom packages, which used to be the case). ports does not provide a unique feature here, and if it is required to manually patch software in order to be reasonably quick to address security issues, that's a bug not a feature.
xenophonf 11y ago
TL;DR: use pkgng. I begin to feel like a broken record saying this again and again, but: ports is a clever tool, but it is not a solution to the problem of package management. Of course the ports tree itself is just package build infrastructure. Those makefiles aren't a fully realized package management system. No one disputes that. Prior to pkgng the answer to all of your problems (which basically boil down to wanting an apt- or yum-like user experience) was to create your own private binary package repo using tinderbox and manage installed packages using something like "portupgrade -PP" or portmaster. If you didn't need anything built with non-default options, you could use portupgrade/portmaster along with the public binary package repos, although I don't know how well maintained these were. If you didn't want to bother with tinderbox (and this is the route I took), you would use portupgrade or portmaster to handle all package management, with all packages built locally from source (ports tree). This latter approach worked well for me even though package building took a while, since all of my FreeBSD systems were servers and each only had a limited set of packages installed. Of course it doesn't scale. I was rather under the impression that most enterprise-scale FreeBSD installations went the tinderbox route. So, I think I need to stop complaining about it, and start asking the right questions of BSD users. Why is ports such a fiercely defended thing when it has so many weaknesses compared to modern package managers? Does pkgng actually solve these problems (I can't really tell, it seems to be a meta-package system that still sits atop ports)? I'm going to answer your second question first: use pkgng. Seriously. It's a complete binary package management system. It works just like apt or yum. The binary package repos are well maintained. Most everything people usually want is in there. I can't speak for other BSD sysadmins, but I love being able to customize package builds to meet my needs. Sure, it's possible to build one's own debs or rpms or what have you, but I think poudriere makes maintaining a private package repo much simpler than something like Spacewalk. And in fact, that's how I do things nowadays. I maintain my own private package repo with everything built to my liking and updated my schedule.

throwaway2048 11y ago

Should be noted that OpenBSD has package tools and management much more along the lines of apt or yum. Ports on openbsd build into packages so you dont get a weirdly broken system if you do need to dip into ports for some reason (although the general recommendation is to use binary packages)

protomyth 11y ago
I don't think I've used ports on FreeBSD and have stuck with pkg. PC-BSD has a pretty extensive package management system[1], and and NetBSD's package management is used by multiple operating systems[2]. Ports isn't really used unless you have some pretty specific needs. 1) http://web.pcbsd.org/doc-archive/10.1.2/html/update.html http://web.pcbsd.org/doc-archive/10.1.2/html/update.html 2) http://www.netbsd.org/docs/software/packages.html http://www.netbsd.org/docs/software/packages.html

vezzy-fnord 11y ago

Library mismatches, dependency conflicts and progressive destabilization/entropy (system package configurations approaching global failure conditions like boot stalling as time passes and upgrades are made) were all frequent experiences I had on dpkg-based systems with the APT frontend that ultimately made me return to Slackware and its spartan package management where I have since experienced nothing of the sort. This is likely a YMMV issue.

marios 11y ago

OpenBSD has both binary packages and ports. Actually, the packages are built from ports, so if you want to customize a package, you checkout the ports tree, adjust the Makefile, add patches or whatever it is you need, and then you get a nice package that you can use on other OpenBSD installations (provided they are running the same version). This is similar to what one does with, say, APT (apt-get {build-dep, source} <pkg>; modify; dpkg-builpackage ...). When you get install a release, you set a package source (/etc/pkg.conf is more or less the equivalent of /etc/apt/*), and then you just pkg_add <pkg>. Package versions are fixed, pretty much the same way you get when you run debian stable. There is one major difference though: OpenBSD is an OS, while Debian is a distribution. This means that debian will install the package files relatively to /, while OpenBSD will install them relatively to /usr/local. It may sound stupid if you've only used Linux before, but it makes sense: files in /usr/bin, /usr/sbin,... are files that were shipped with the OS. What lives in /usr/local is what you installed on top of it. Other than that, I don't see any major difference. You occasionally get some libraries that are out of sync with a package, but that happens on -current and it's usually that the package was not rebuilt yet on the mirror you're using. (Ever run Debian sid ? Well, it's the same thing) What I'm saying is that OpenBSD uses ports effectively and in day to day usage I see no difference to APT-based distributions. I've never used it, but I hear very good things about pkgsrc (developed by NetBSD but it can also be used elsewhere. AFAIK, Joyent uses it and some people run it on Linux too). From what I can see on the homepage, it provides both sources and binaries. Until pkgng was introduced, FreeBSD was lagging behind but from what I can tell, it seems the situation has improved. What "weaknesses" related to ports do you have in mind ?

SwellJoe 11y ago
"What "weaknesses" related to ports do you have in mind ?" The ones I've mentioned (and mentioned in the article we're commenting on): - Library mismatches leading to broken packages, sometimes on a freshly installed system. I've always heard outright denial that this happens, but I've literally never installed a FreeBSD system that didn't end up with something broken by the time everything needed for a fully functioning web hosting system was installed. It's like there's Stockholm syndrome happening, or something. - Poor dependency resolution, and weak validation tools, that requires manual intervention to prevent or repair those breakages which are bound to occur. Other complaints I have about ports: - No reasonable mechanism for offering third party package repositories. - Default ports are usually anemic, missing very basic functionality, guaranteeing you have to build some things from source in order to get a really functional version of, say, Apache. Now, you have to make your own means of distributing these custom builds across your systems, without any reasonable means of setting up your own repository (you can duplicate the whole ports tree, and merge your changes, which is ridiculous, and the tools to help with this are weak). Really, the biggest failings are in dependency specification and resolution. I don't know if it is a process failure (e.g., nobody is taking the time to specify things precisely enough to end up with a working package in every circumstance) or failure of the system (e.g., the system doesn't provide the means to specify things precisely enough to insure a functional system), or a combination of the two. But, the end result is that it is frightfully easy to end up with broken software, and often requires a lot of knowledge and human interaction to fix it. Again, providing binaries is fine and all (and, I think, mandatory for large automated deployments), but is not the basis of the problems I've experienced with ports. The fragility of ports is my problem with it, not the surface level differences between how things are installed and updated.

2trill2spill 11y ago

> So, I think I need to stop complaining about it, and start asking the right questions of BSD users. Why is ports such a fiercely defended thing when it has so many weaknesses compared to modern package managers? First theres no dichotomy here. A person can use ports and pkgng. Secondly there's many nice thing about ports. If your organization has custom patches or compile options for common software, you can use the port system. Also I think a more important use of ports, is security fixes. When OpenSSL drops it's bug of the week you can patch openssl and rebuild the port tree. Then you can push the new, patched binaries into production. A sysadmin can do this all without relying on the package maintainer, ie no waiting on Redhat to push out new rpms. This can all be done in under an hour depending on the performance of the build server and automation processes in place. > Does pkgng actually solve these problems (I can't really tell, it seems to be a meta-package system that still sits atop ports)? Yes. > and building from source on a production system is silly and potentially dangerous Well of course you don't build directly on the production box. That's what a build server is for. You can use poudriere to build packages from the ports tree and then run your own package repo and have your production boxes pull from your private package repo.

SwellJoe 11y ago
"Also I think a more important use of ports, is security fixes. When OpenSSL drops it's bug of the week you can patch openssl and rebuild the port tree. Then you can push the new, patched binaries into production. A sysadmin can do this all without relying on the package maintainer, ie no waiting on Redhat to push out new rpms." I can't remember the last time I was aware of a security problem before Red Hat (and CentOS) had rolled out a fixed package. I usually know about security issues when I see a slew of update notifications from my servers (even though I'm on several security related mailing lists). Realistically, I rely on my OS vendor to provide my security fixes in a timely manner. If my systems security relied on me being able to apply patches manually in a timely manner, it would be a disaster waiting to happen. And, it is a big pile of labor I don't want on my plate. Besides, nothing stops one from installing the source package (whether RPM or deb), adding the patch, rebuilding, and pushing to your local yum or apt repository. That workflow is entirely reasonable on Linux, as well (historically easier, since maintaining a third party repo was so much easier for yum, though apt is a bit more cantankerous; it looks like FreeBSD finally supports multiple repositories and you don't need to mirror the whole ports tree just to provide a handful of custom packages, which used to be the case). ports does not provide a unique feature here, and if it is required to manually patch software in order to be reasonably quick to address security issues, that's a bug not a feature.

xenophonf 11y ago

TL;DR: use pkgng. I begin to feel like a broken record saying this again and again, but: ports is a clever tool, but it is not a solution to the problem of package management. Of course the ports tree itself is just package build infrastructure. Those makefiles aren't a fully realized package management system. No one disputes that. Prior to pkgng the answer to all of your problems (which basically boil down to wanting an apt- or yum-like user experience) was to create your own private binary package repo using tinderbox and manage installed packages using something like "portupgrade -PP" or portmaster. If you didn't need anything built with non-default options, you could use portupgrade/portmaster along with the public binary package repos, although I don't know how well maintained these were. If you didn't want to bother with tinderbox (and this is the route I took), you would use portupgrade or portmaster to handle all package management, with all packages built locally from source (ports tree). This latter approach worked well for me even though package building took a while, since all of my FreeBSD systems were servers and each only had a limited set of packages installed. Of course it doesn't scale. I was rather under the impression that most enterprise-scale FreeBSD installations went the tinderbox route. So, I think I need to stop complaining about it, and start asking the right questions of BSD users. Why is ports such a fiercely defended thing when it has so many weaknesses compared to modern package managers? Does pkgng actually solve these problems (I can't really tell, it seems to be a meta-package system that still sits atop ports)? I'm going to answer your second question first: use pkgng. Seriously. It's a complete binary package management system. It works just like apt or yum. The binary package repos are well maintained. Most everything people usually want is in there. I can't speak for other BSD sysadmins, but I love being able to customize package builds to meet my needs. Sure, it's possible to build one's own debs or rpms or what have you, but I think poudriere makes maintaining a private package repo much simpler than something like Spacewalk. And in fact, that's how I do things nowadays. I maintain my own private package repo with everything built to my liking and updated my schedule.