9 ms·

Privatization Means Rebuilds

30 points by protomyth 11y ago

ChuckMcM 11y ago
Cached version : http://webcache.googleusercontent.com/search?q=cache:_t5yQSLC0xIJ:https://www.dragonflydigest.com/2016/01/27/17547.html&num=1&hl=en&gl=us&strip=1&vwsrc=0 http://webcache.googleusercontent.com/search?q=cache:_t5yQSL... And the summary is, separate "system" requirements/dependencies from those that packages depend on so that rebuilding a package doesn't break the system.
- protomyth 11y ago
  I think it is the opposite way around, so that rebuilding the system doesn't break a package. Basically, they can switch system libraries or delete them entirely without the packages being affected since they will be using the non-private libraries from packages / ports.
teac2019 11y ago
https://www.youtube.com/watch?v=UMnC3Nwif1k https://www.youtube.com/watch?v=UMnC3Nwif1k
SwellJoe 11y ago
I still can't understand why several BSDs so stubbornly cling to ports as a "solution" to package management. I ran into this problem, and variants of it, over and over again when trying to support FreeBSD in our products, in the past. Though pkgng may resolve some of my problems with package management on FreeBSD (it at least allows third party repositories, the lack of which was a big pain point for me in the past), I haven't tried it yet, and from what I can tell FreeBSD still favors ports, and several other BSDs are still using ports exclusively. I've even seen this library mismatch happen on a couple of occasions in a fresh install, where installing a variety of ports leads to one or some of them being broken immediately after install because several packages called for different versions of libraries. And, the longer a BSD system is in service, the more likely some portupgrade, or installation of a new package, is to break some other thing on the system, requiring far reaching rebuilds (and sometimes, manual intervention, because the dependencies specified aren't sufficient to allow the system to automatically fix itself). I begin to feel like a broken record saying this again and again, but: ports is a clever tool, but it is not a solution to the problem of package management. There's so much to love about the BSDs, but lack of effective package management has always made it a non-starter for any purpose, for me. Effective package management is not esoteric or bleeding edge or experimental stuff, at this point. deb-based systems had apt 20+ years ago, RPM based systems had yum ~12 years ago. Slapping a band aid on the darned thing doesn't fix it. And, so many conversations about packages devolve into ports proponents missing the point, and saying, "Well, you can install binary versions of ports, so it's fine." When the availability of binary packages is utterly orthogonal to the problems I'm talking about (though also a big deal, and building from source on a production system is silly and potentially dangerous). So, I think I need to stop complaining about it, and start asking the right questions of BSD users. Why is ports such a fiercely defended thing when it has so many weaknesses compared to modern package managers? Does pkgng actually solve these problems (I can't really tell, it seems to be a meta-package system that still sits atop ports)?
- throwaway2048 11y ago
  Should be noted that OpenBSD has package tools and management much more along the lines of apt or yum. Ports on openbsd build into packages so you dont get a weirdly broken system if you do need to dip into ports for some reason (although the general recommendation is to use binary packages)
  - protomyth 11y ago
    I don't think I've used ports on FreeBSD and have stuck with pkg. PC-BSD has a pretty extensive package management system[1], and and NetBSD's package management is used by multiple operating systems[2]. Ports isn't really used unless you have some pretty specific needs. 1) http://web.pcbsd.org/doc-archive/10.1.2/html/update.html http://web.pcbsd.org/doc-archive/10.1.2/html/update.html 2) http://www.netbsd.org/docs/software/packages.html http://www.netbsd.org/docs/software/packages.html
- vezzy-fnord 11y ago
  Library mismatches, dependency conflicts and progressive destabilization/entropy (system package configurations approaching global failure conditions like boot stalling as time passes and upgrades are made) were all frequent experiences I had on dpkg-based systems with the APT frontend that ultimately made me return to Slackware and its spartan package management where I have since experienced nothing of the sort. This is likely a YMMV issue.
- marios 11y ago
  OpenBSD has both binary packages and ports. Actually, the packages are built from ports, so if you want to customize a package, you checkout the ports tree, adjust the Makefile, add patches or whatever it is you need, and then you get a nice package that you can use on other OpenBSD installations (provided they are running the same version). This is similar to what one does with, say, APT (apt-get {build-dep, source} <pkg>; modify; dpkg-builpackage ...). When you get install a release, you set a package source (/etc/pkg.conf is more or less the equivalent of /etc/apt/*), and then you just pkg_add <pkg>. Package versions are fixed, pretty much the same way you get when you run debian stable. There is one major difference though: OpenBSD is an OS, while Debian is a distribution. This means that debian will install the package files relatively to /, while OpenBSD will install them relatively to /usr/local. It may sound stupid if you've only used Linux before, but it makes sense: files in /usr/bin, /usr/sbin,... are files that were shipped with the OS. What lives in /usr/local is what you installed on top of it. Other than that, I don't see any major difference. You occasionally get some libraries that are out of sync with a package, but that happens on -current and it's usually that the package was not rebuilt yet on the mirror you're using. (Ever run Debian sid ? Well, it's the same thing) What I'm saying is that OpenBSD uses ports effectively and in day to day usage I see no difference to APT-based distributions. I've never used it, but I hear very good things about pkgsrc (developed by NetBSD but it can also be used elsewhere. AFAIK, Joyent uses it and some people run it on Linux too). From what I can see on the homepage, it provides both sources and binaries. Until pkgng was introduced, FreeBSD was lagging behind but from what I can tell, it seems the situation has improved. What "weaknesses" related to ports do you have in mind ?
Animats 11y ago
I didn't know that BSD now had distros.
- 2trill2spill 11y ago
  It's not a distro, dragonflyBSD is it's own Operating system. DragonflyBSD split from FreeBSD when they disagreed on how to do SMP. But I guess PC-BSD could be considered a distro.
  - groovy2shoes 11y ago
    The "D" in "BSD" stands for "distribution"...
stock_toaster 11y ago
This is great. A good example of why this is great, is openssl. Because ports/packages build with the base openssl, a security team supporting a release usually has to maintain an abi compatible version in base. If ports/packages simply used their a packaged of openssl, it would make it much easier for ports/packages to update in lockstep, as well as the base system to update on its own schedule (update to newer version and simply update any outdated api usages in base as well -- and for example to use libressl instead). I think BSD's exporting so many libraries as part of the base install made much more sense in the time with small hard drives, but maybe not as much today.
- xenophonf 11y ago
  The following isn't exactly right: Because ports/packages build with the base openssl, a security team supporting a release usually has to maintain an api compatible version in base. Forcing ports to compile against the security/openssl port instead of the base version is pretty easy. You add the following knob to /etc/make.conf, to the port's Makefile, or to the make(1) command line: WITH_OPENSSL_PORT=yes The reason the security team has to maintain an API/ABI-compatible release of OpenSSL is because FreeBSD releases make that guarantee for all base system libraries---and the reason OpenSSL is in the base system in the first place is because other things in base like OpenSSH and Heimdal require it. This guarantee lets me build packages on FreeBSD 10.0 that work fine on FreeBSD 10.2 (and, generally, vice versa).
  - stock_toaster 11y ago
    > Forcing ports to compile against the security/openssl port > instead of the base version is pretty easy. Yes. I do this currently, and it works great. But not everyone does, so the security team has to keep abi compatibility. > The reason the security team has to maintain an API/ABI-compatible release > of OpenSSL is because FreeBSD releases make that guarantee for all base > system libraries---and the reason OpenSSL is in the base system in the > first place is because other things in base like OpenSSH and Heimdal > require it. Head scratching That was...kind of my whole point. Anyway, openssl is planned to become private, so I just need to wait a bit (maybe Allan will get it into FreeBSD-11? see[1]). [1]: https://wiki.freebsd.org/OpenSSL/Base https://wiki.freebsd.org/OpenSSL/Base

30 points by protomyth 11y ago

Cached version : http://webcache.googleusercontent.com/search?q=cache:_t5yQSLC0xIJ:https://www.dragonflydigest.com/2016/01/27/17547.html&num=1&hl=en&gl=us&strip=1&vwsrc=0 http://webcache.googleusercontent.com/search?q=cache:_t5yQSL... And the summary is, separate "system" requirements/dependencies from those that packages depend on so that rebuilding a package doesn't break the system.

protomyth 11y ago
I think it is the opposite way around, so that rebuilding the system doesn't break a package. Basically, they can switch system libraries or delete them entirely without the packages being affected since they will be using the non-private libraries from packages / ports.

teac2019 11y ago

https://www.youtube.com/watch?v=UMnC3Nwif1k https://www.youtube.com/watch?v=UMnC3Nwif1k

SwellJoe 11y ago

I still can't understand why several BSDs so stubbornly cling to ports as a "solution" to package management. I ran into this problem, and variants of it, over and over again when trying to support FreeBSD in our products, in the past. Though pkgng may resolve some of my problems with package management on FreeBSD (it at least allows third party repositories, the lack of which was a big pain point for me in the past), I haven't tried it yet, and from what I can tell FreeBSD still favors ports, and several other BSDs are still using ports exclusively. I've even seen this library mismatch happen on a couple of occasions in a fresh install, where installing a variety of ports leads to one or some of them being broken immediately after install because several packages called for different versions of libraries. And, the longer a BSD system is in service, the more likely some portupgrade, or installation of a new package, is to break some other thing on the system, requiring far reaching rebuilds (and sometimes, manual intervention, because the dependencies specified aren't sufficient to allow the system to automatically fix itself). I begin to feel like a broken record saying this again and again, but: ports is a clever tool, but it is not a solution to the problem of package management. There's so much to love about the BSDs, but lack of effective package management has always made it a non-starter for any purpose, for me. Effective package management is not esoteric or bleeding edge or experimental stuff, at this point. deb-based systems had apt 20+ years ago, RPM based systems had yum ~12 years ago. Slapping a band aid on the darned thing doesn't fix it. And, so many conversations about packages devolve into ports proponents missing the point, and saying, "Well, you can install binary versions of ports, so it's fine." When the availability of binary packages is utterly orthogonal to the problems I'm talking about (though also a big deal, and building from source on a production system is silly and potentially dangerous). So, I think I need to stop complaining about it, and start asking the right questions of BSD users. Why is ports such a fiercely defended thing when it has so many weaknesses compared to modern package managers? Does pkgng actually solve these problems (I can't really tell, it seems to be a meta-package system that still sits atop ports)?

throwaway2048 11y ago
Should be noted that OpenBSD has package tools and management much more along the lines of apt or yum. Ports on openbsd build into packages so you dont get a weirdly broken system if you do need to dip into ports for some reason (although the general recommendation is to use binary packages)
- protomyth 11y ago
  I don't think I've used ports on FreeBSD and have stuck with pkg. PC-BSD has a pretty extensive package management system[1], and and NetBSD's package management is used by multiple operating systems[2]. Ports isn't really used unless you have some pretty specific needs. 1) http://web.pcbsd.org/doc-archive/10.1.2/html/update.html http://web.pcbsd.org/doc-archive/10.1.2/html/update.html 2) http://www.netbsd.org/docs/software/packages.html http://www.netbsd.org/docs/software/packages.html
vezzy-fnord 11y ago
Library mismatches, dependency conflicts and progressive destabilization/entropy (system package configurations approaching global failure conditions like boot stalling as time passes and upgrades are made) were all frequent experiences I had on dpkg-based systems with the APT frontend that ultimately made me return to Slackware and its spartan package management where I have since experienced nothing of the sort. This is likely a YMMV issue.
marios 11y ago
OpenBSD has both binary packages and ports. Actually, the packages are built from ports, so if you want to customize a package, you checkout the ports tree, adjust the Makefile, add patches or whatever it is you need, and then you get a nice package that you can use on other OpenBSD installations (provided they are running the same version). This is similar to what one does with, say, APT (apt-get {build-dep, source} <pkg>; modify; dpkg-builpackage ...). When you get install a release, you set a package source (/etc/pkg.conf is more or less the equivalent of /etc/apt/*), and then you just pkg_add <pkg>. Package versions are fixed, pretty much the same way you get when you run debian stable. There is one major difference though: OpenBSD is an OS, while Debian is a distribution. This means that debian will install the package files relatively to /, while OpenBSD will install them relatively to /usr/local. It may sound stupid if you've only used Linux before, but it makes sense: files in /usr/bin, /usr/sbin,... are files that were shipped with the OS. What lives in /usr/local is what you installed on top of it. Other than that, I don't see any major difference. You occasionally get some libraries that are out of sync with a package, but that happens on -current and it's usually that the package was not rebuilt yet on the mirror you're using. (Ever run Debian sid ? Well, it's the same thing) What I'm saying is that OpenBSD uses ports effectively and in day to day usage I see no difference to APT-based distributions. I've never used it, but I hear very good things about pkgsrc (developed by NetBSD but it can also be used elsewhere. AFAIK, Joyent uses it and some people run it on Linux too). From what I can see on the homepage, it provides both sources and binaries. Until pkgng was introduced, FreeBSD was lagging behind but from what I can tell, it seems the situation has improved. What "weaknesses" related to ports do you have in mind ?

Animats 11y ago

I didn't know that BSD now had distros.

2trill2spill 11y ago
It's not a distro, dragonflyBSD is it's own Operating system. DragonflyBSD split from FreeBSD when they disagreed on how to do SMP. But I guess PC-BSD could be considered a distro.
- groovy2shoes 11y ago
  The "D" in "BSD" stands for "distribution"...

stock_toaster 11y ago

This is great. A good example of why this is great, is openssl. Because ports/packages build with the base openssl, a security team supporting a release usually has to maintain an abi compatible version in base. If ports/packages simply used their a packaged of openssl, it would make it much easier for ports/packages to update in lockstep, as well as the base system to update on its own schedule (update to newer version and simply update any outdated api usages in base as well -- and for example to use libressl instead). I think BSD's exporting so many libraries as part of the base install made much more sense in the time with small hard drives, but maybe not as much today.

xenophonf 11y ago
The following isn't exactly right: Because ports/packages build with the base openssl, a security team supporting a release usually has to maintain an api compatible version in base. Forcing ports to compile against the security/openssl port instead of the base version is pretty easy. You add the following knob to /etc/make.conf, to the port's Makefile, or to the make(1) command line: WITH_OPENSSL_PORT=yes The reason the security team has to maintain an API/ABI-compatible release of OpenSSL is because FreeBSD releases make that guarantee for all base system libraries---and the reason OpenSSL is in the base system in the first place is because other things in base like OpenSSH and Heimdal require it. This guarantee lets me build packages on FreeBSD 10.0 that work fine on FreeBSD 10.2 (and, generally, vice versa).
- stock_toaster 11y ago
  > Forcing ports to compile against the security/openssl port > instead of the base version is pretty easy. Yes. I do this currently, and it works great. But not everyone does, so the security team has to keep abi compatibility. > The reason the security team has to maintain an API/ABI-compatible release > of OpenSSL is because FreeBSD releases make that guarantee for all base > system libraries---and the reason OpenSSL is in the base system in the > first place is because other things in base like OpenSSH and Heimdal > require it. Head scratching That was...kind of my whole point. Anyway, openssl is planned to become private, so I just need to wait a bit (maybe Allan will get it into FreeBSD-11? see[1]). [1]: https://wiki.freebsd.org/OpenSSL/Base https://wiki.freebsd.org/OpenSSL/Base