44 ms·

I am arguing about this on FreeBSD forums - from an end user perspective. I think the benefits of saving disk space by sharing libraries do not justify the inco

I am arguing about this on FreeBSD forums - from an end user perspective. I think the benefits of saving disk space by sharing libraries do not justify the inconveniences we, desktop BSD and Linux users, obtain from being forced to disrupt and upgrade hundreds of installed software packages, just because one single desired upgrade requires to pull its dependencies. The whole ecosystem falls down like a card house - once in a while I have to say "screw it" and auto-update more than a thousand packages, praying my Python and other projects survive, just because I want a browser update or a security fix. I wonder if there are others who support this opinion that desktop Unix has very complicated future unless complex apps will begin to bundle their own libraries.

progval 7y ago
> auto-update more than a thousand packages, praying my Python and other projects survive, just because I want a browser update You may want to check out Nix. It's a package manager which isolates each program's dependencies; so you can have multiple versions of the same package. > or a security fix having a centralized repository like this helps in case a it's a library that needs a security fix, because you only need a single update to apply the fix to all applications. Applications that bundle their own deps (including Docker images) will need to publish their own update with the updated version of the lib.
- l0b0 7y ago
  > You may want to check out Nix. Nix is a fantastic concept, and I hope it takes over the world. But the NixOS packages are a mess. I tried it for a few months last year before giving up after several packages and even whole collections of packages became unusable even in the stable repository. The repository needs some serious reworking before Nix can really shine.
  - zrm 7y ago
    It's really trading one problem for a different one. The reason traditional distributions only package one version of a given library and make everything use that same version is so they only have to maintain one version of the library. If you allow every application to choose its own version then they all choose a different one, which means someone then has to continue to maintain every different version of the library separately. Doing that isn't much if any of a reduction in workload compared with getting every application to use the version the distribution ships, and is much more likely to end up with a situation where dozens of versions of the same library exist and half of them are broken in some way. The better solution is for libraries to only break compatibility between major versions. Then the packager can ship some suitably recent minor version for each major version of the library, at most two or three major versions are supported at any given time which keeps the maintenance overhead feasible, and every application can successfully link against one of those major versions or another.
    - avar 7y ago
      It doesn't entail any of that. Traditional distributions like Debian allow you to install any number of library combinations, they just do so badly by requiring you to do so on different installations updated at different times. Supporting the presence of multiple library combinations doesn't mean that you "support" all version combinations in the sense of supporting all combinations of their versions being used in the wild. It just means when users upgrade they can do so gradually for some sets of packages at a time. You can emulate this on traditional distributions by having a single VM image you use for your browser, then cloning the image, updating packages, and using that one for your E-Mail etc. Having a mechanism like this should mean less support is needed from the distribution, because system-wide breakages are less likely to occur. You'll get the same bugs, but users can easily back out say an OpenSSL version with a security fix for the 5% of packages it breaks with, while retaining the fix for their browser & other high-risk packages. For example, I currently can't upgrade my firefox version on Debian testing because it ">=" depends on a version of a couple of libraries that some 100-200 other packages on my system don't like. In Debian testing this sort of thing happens occasionally and will get fixed sooner than later, but that it happens at all is just an emergent property of how the versions are managed. If I were to run the latest firefox with those new versions and not touch the other packages until they're recompiled both myself and the package maintainers would be exposed to fewer bugs, not less. I'd get a bugfixed firefox today, and they wouldn't need to deal with bug reports about a broken upgrade, or firefox bugs in an older version users like me can't upgrade past simply because firefox happens to share the use of a popular OS library.
      - zrm 7y ago
        > Supporting the presence of multiple library combinations doesn't mean that you "support" all version combinations in the sense of supporting all combinations of their versions being used in the wild. It's not about combinations. It's about, there's a serious bug in the library and now there are 79 different actively used versions of it that need to be patched instead of 2, which means half of them never get fixed. > You'll get the same bugs, but users can easily back out say an OpenSSL version with a security fix for the 5% of packages it breaks with, while retaining the fix for their browser & other high-risk packages. You'll get the same bugs and then the 5% of packages a security update breaks are broken until they're fixed, which should happen promptly because broken packages should be very high priority. Is it really better in that short time for the other 5% of applications to "work" but be insecure and get your system compromised because the new vulnerability is being actively exploited? The real solution is for sufficient testing to be done during the embargo period so that by the time the new version goes wide it doesn't actually break anything. > For example, I currently can't upgrade my firefox version on Debian testing because it ">=" depends on a version of a couple of libraries that some 100-200 other packages on my system don't like. Right, that's a problem. It's valid to need version >= 5.6 of some library, but something has gone wrong if some other package specifically needs version 5.5 -- version 5.6 should be able to satisfy that dependency. Or if it can't then "5.6" should be 6.0 and it should be possible to have 5.x and 6.x installed together, but that should happen only rarely so that at most two or three mutually incompatible versions are in active use at once. In general the latest version should be able to satisfy applications expecting older versions and then you need only a suitably recent version and nothing else, and the lack of that is the real issue.
  - millettjon 7y ago
    I've been using it for 2 years with very few problems.
- danieldk 7y ago
  You may want to check out Nix. It's a package manager which isolates each program's dependencies; so you can have multiple versions of the same package. In principle true, yes. But it comes with a cost. Even the nixpkgs Firefox maintainers are considering to only ship Firefox ESR starting with NixOS 20.03: https://discourse.nixos.org/t/firefox-on-19-09-could-be-marked-as-insecure-or-downgraded-to-esr-68/4577 https://discourse.nixos.org/t/firefox-on-19-09-could-be-mark... https://github.com/NixOS/nixpkgs/pull/72126 https://github.com/NixOS/nixpkgs/pull/72126
  - progval 7y ago
    I won't blame them. Firefox is a huge piece of software, and is updating every month. It's a bit shitty of Mozilla to expect every single distribution (most of which are run by volunteers) to package all their new dependencies and check everything works every single month.
    - rusk 7y ago
      This sounds like ”move fast and break things” writ large, and a salutary lesson in the flaws of this approach.
    - jcelerier 7y ago
      but you can just use Mozilla's official binary packages, which are tested to work with a specific set of libraries, and I think are built with higher optimization levels (O3/pgo) than what distros conservatively do (O2) because of cargo-culting early-2000 gentoo forums (that was the case a few years ago, not 100% sure it is still the case today as I could not find easily the current release flags).
      - progval 7y ago
        Mozilla doesn't provide binary packages for all architectures and OSs supported by Nix/NixOS Maintainers also sometimes need to patch software to conform to their distribution's norm. For example, Debian patches it to accept extensions installed with APT. And I'm glad distribution maintainers always recompile software themselves, it helps making sure: 1. binaries are really made from the open source code (I know, it's not bullet-proof) 2. it's actually humanly possible to actually compile it from source (my pet peeve in this area is Kafka: it's used everywhere, but no one seems to know how to actually fully compile it from sources anymore)
- jrockway 7y ago
  Is this kind of like Ubuntu's "snap" thing? (I know snap involves cgroups, which aren't strictly necessary to give Firefox a different version of rust than everything else... but it seems widely deployed and easy to use, at least if you're on Ubuntu.)
- xvilka 7y ago
  Guix[1] also. It's written in Scheme but also solves the bootstrapping problem with minimal seed[2]. [1] http://guix.gnu.org/ http://guix.gnu.org/ [2] https://guix.gnu.org/manual/en/html_node/Reduced-Binary-Seed-Bootstrap.html https://guix.gnu.org/manual/en/html_node/Reduced-Binary-Seed...
Avamander 7y ago
> praying my Python and other projects survive, just because I want a browser update or a security fix. You mean you aren't familiar with Python's virtual environment system exactly intended for isolating development dependencies from system ones but you're blaming the distribution. Please.
- sunaurus 7y ago
  None of my virtualenvs (on Linux or on OS X) survived the 3.7 -> 3.8 upgrade. Are you saying they shouldn't have broken?
  - pmontra 7y ago
    Virtualenv lets you use only a Python interpreter that's already installed on your system. Usually the choice is between the latest v2 and v3 interpreter, at least on Debian derivatives. Every virtualenv is autoupgraded each time the OS upgrades its Pythons. If you want to use multiple versions of Python and pin them, you should use something like asdf [1] and its Python plugin [2]. It's a version manager with support for many languages and more. I used it for Elixir, Node, PHP, Ruby and even PostgreSQL (I need different database versions in different projects.) I never used it for Python because I'm OK with being on the latest version so far. With asdf I would expect to do something like this asdf install python 3.7.5 mkdir my375project cd my375project asdf local python 3.7.5 # pick the version to use here python3 -m venv py375 --python=~/.asdf/installs/python/3.7.5/bin/python ~/.virtualenvs/py375/bin/activate This virtualenv is going to stay on 3.7.5 forever. [1] https://github.com/asdf-vm/asdf https://github.com/asdf-vm/asdf [2] https://github.com/danhper/asdf-python https://github.com/danhper/asdf-python
- rleigh 7y ago
  Virtualenvs don't protect you completely. If any of the modules in the virtualenv depend upon libraries on the host system, and a lot of Python modules do either directly or indirectly, then there is a chance of breakage if any of the ABIs of these modules break or there's an incompatible ABI between any of the modules. Even virtualenvs themselves can cause the breakage. Take "pylibtiff". It embeds a copy of libtiff. But, it's old and incompatible with other libraries on the base system. This can be libraries it depends upon, or it could be another copy of the same library. Either way, virtualenvs can cause more problems than they solve.
smartmic 7y ago
Hm. For me this is a question about interface design (vs. implementation). IMHO Unix programming philosophy is a lighthouse how to solve this commendable. In an ideal world you should have to upgrade base program and dependent libraries only if their interfaces have changed. EDIT: And of course for security reasons - but then you only upgrade the affected player and not huge parts of your installed base. What I want to say - this is all about the ruling design philosophy.
floatingatoll 7y ago
RedHat’s “streams” model will certainly do a much better job of handling this than the other distributions do today. I hope that the need for having multiple parallel versions of a dependency coexist is incorporated into the other distros, because I’ve lost a lot of sanity this past two decades to the assumption that “one installed version should be enough for anybody” on Linux and BSD servers.
- ethbro 7y ago
  FYI, for those like me whose last memory of RedHat packaging was pre-yum: https://developers.redhat.com/blog/2018/11/15/rhel8-introducing-appstreams/ https://developers.redhat.com/blog/2018/11/15/rhel8-introduc...
- offmycloud 7y ago
  I hope so, but Fedora is dragging their feet putting out a patched Firefox package on Fedora 30, which is still suppoted.
  - cesarb 7y ago
    You can check the status for the firefox package in Fedora at https://admin.fedoraproject.org/updates/ https://admin.fedoraproject.org/updates/ which shows firefox-72.0.1-1.fc30 as "testing" (that is, you can install through "yum --enablerepo=updates-testing update firefox"). The reason for it not being promoted to stable yet is, according to that page, "disabling automatic push to stable due to negative karma" - that is, the update was marked as broken by a couple of people.
- seanparsons 7y ago
  Already handled in NixOS by Nix.
- Qerub 7y ago
  AppStream does not allow installation of multiple versions of the same app AFAIK. I believe this is what they refer to in the clumsy sentence "The one disadvantage of Application Streams from SCLs is that no two streams can be installed at the same time in to the same userspace. However, in our experience, this is not a common use case and is better served using containerization or virtualization to provide a second userspace." in the linked article in the sibling comment.
markstos 7y ago
Complex apps can bundle their own dependencies-- that's what the Flatpak and Snap package formats do. You can also run Firefox in a Docker or LXD container with it's dependencies. By sharing the X11 or Wayland socket with the docker, the apps can appear on your main desktop.
- jackhalford 7y ago
  none of those solutions apply the BSDs though
  - LeoPanthera 7y ago
    Can you run graphical applications inside a BSD jail?
    - napsterbr 7y ago
      You can with SSH and X11 forwarding. Whether that's practical is a different question and may vary according to the application and user
      - toast0 7y ago
        You wouldn't need to use SSH, just linking/bind mounting the X socket and xauth into the jail (plus/minus whatever shm stuff).
    - beefhash 7y ago
      Pedantic note: Jails are not a BSD concept, they're a FreeBSD concept. OpenBSD has no notion of jails.
    - dddddaviddddd 7y ago
      Yes. Here's an example for Firefox on FreeBSD: https://wiki.freebsd.org/VladimirKrstulja/Guides/JailingGUIApplications https://wiki.freebsd.org/VladimirKrstulja/Guides/JailingGUIA...
  - 7y ago
ChickeNES 7y ago
I wish NeXTsteps's app bundles had caught on outside of OSX.
- hoistbypetard 7y ago
  Me too. Especially as combined with fat binaries in the OS X rosetta years. It'd make it a lot easier to package something that would run on arm, arm64 and amd64 in a more accessible way for users.
- stock_toaster 7y ago
  AppImage[1] kind of works that way. I'm surprised it hasn't caught on more, tbh. I don't think it supports multiple architectures like app bundles can (fat bundles) though. [1]: https://appimage.org https://appimage.org
  - lokedhs 7y ago
    One of my applications is distributed through Appimage. It works really well. I really feel it should support containerisation in a similar way as Flatpak does. In my opinion, the idea that any application on your system can can touch all your files and access all your other applications is really problematic. You're always just one broken (intentionally or not) application form having your entire system compromised. As best as I can tell, the only solutions that even tries to address this is Qubes OS and Flatpak.
- pjmlp 7y ago
  On Windows they are called xcopy install, and they have also caught on mobile OSes.
sandov 7y ago
I'll go one step further: The whole "centralized, trusted repository that has all your apps" system is wrong at a fundamental level. The way shared libraries are used in Linux is built upon the assumption that package managers and centralized repositories are the right way to do things.
- Analemma_ 7y ago
  I agree, and I think that people are too emotionally invested in the package manager concept to back out now. I mean, for years Linux proponents have been touting it as the key advantage over software distribution on Macs/PCs.
  - hoistbypetard 7y ago
    The existence and popularity of homebrew, macports and fink (on mac) and ninite, chocolatey, cygwin, mingw and nuget (on windows) would suggest that the concept appeals to mac and windows users also.
  - microcolonel 7y ago
    I mean, it is a key advantage, on ArchLinux. If you don't have a rolling release, it's pure pain: you're often just stuck with old versions of software which aren't any more stable than the current stable release.
    - TheCraiggers 7y ago
      It's only pure pain if you want bleeding edge AND stability. Non-rolling releases are just fine if you only need new features once a year.
      - snazz 7y ago
        Non-rolling releases might work with smaller software where your distro maintainers can backport security bugfixes easily, but as soon as your software gets too big and complicated the distro maintainers won't be able to backport fixes themselves and will have to just resort to packaging the latest version of certain packages (like Debian had to do with Chromium and Firefox). Also, for the most part, the "Arch is unstable" thing hasn't been true for a long time. I find that it's far more stable than Debian or Ubuntu LTS on my machine because of newer kernels containing newer hardware drivers.
zozbot234 7y ago
Outside packages should NOT be disrupted given semver. The whole point of using shared objects (dynamically-linked libraries) is so that when a problem arises you can update whatever pieces of code in a centralized, system-wide store and every single one of the projects you use can benefit from the new, up-to-date version. Using the latest version is just the right thing to do.
- stanferder 7y ago
  Does it actually work though? It seems like there is a significant difference between the expectation that semantic versioning should result in no disruption, and the experience of the parent to your comment. I'd be interested to know exactly what went wrong that lead that poster to be pessimistic.
  - jiveturkey 7y ago
    Yes it does actually work and has for decades. The poster is either confused or doing something different, I expect. The specific notice mentions rust dependencies. Rust does not have shared libraries, so a Rust [security] update means all rust binaries must be completely rebuilt. That seems to be part of OpenBSD's concern, and perhaps this has "triggered" the poster. user blackhaz random forum post found: > mariourk, you're definitely not alone. The same breakage happens to FreeBSD desktops as well. I have been vocal around the forum a few times about this. Not every desktop user wants to upgrade all their packages every X months. The way the default repositories are structured, the upgrades are forced onto users. I am a huge proponent of using application bundles, like on Mac OS, as quite often those upgrades break complex desktop apps too. I don't mind upgrades but sometimes I need to stick with a specific version of software, or roll back after something went wrong, and if it came as a bundle with its own dependencies, it wouldn't have to depend on other stuff that is being "force-upgraded" periodically. So his complaint is about the way FBSD handles updates; he is generally incorrect about shared libraries; and his comment is completely OT for this thread.
    - zozbot234 7y ago
      > Rust does not have shared libraries, so a Rust [security] update means all rust binaries must be completely rebuilt. What's the concern with that, exactly? OpenBSD is a security-focused distribution, do they not do reproducible builds?
NelsonMinar 7y ago
Dumb solution with a blunt hammer; why not run Firefox in a container like Docker?
- pferde 7y ago
  I do something like that. I have a separate minimal chroot directory with just Firefox installed and configured, and use systemd-nspawn to start a container. The container is using a snapshot (btrfs) of the chroot directory, which gets deleted when I close the Firefox instance. I use this not as my main browser, but only when I can't avoid visiting "the wild web", meaning unknown or untrusted sites, or even sites that use too much javascript for me to feel safe allowing in my main browser.
rvz 7y ago
Well the reality is, some developers of these complex software complain how their software ported to alternative OSes is difficult to maintain even when the BSD devs would offer to maintain it, which is why the BSDs are always on their own here in terms of porting, testing, packaging and updating the software in question. For example, Chromium. Any BSD developer would know that attempting to upstream their port there is dead in the water. AFAICT, They Chromium devs only care about Win, Mac, Linux and nothing else. Likewise for Firefox, which is why the BSDs don't get official releases. I feel some sympathy for OSes like the BSDs that for "some" software they are limited by the arrogance of other open-source maintainers who would rather go for maintaining the convoluted distros and testing with a huge disintegrated software-stack than testing a single unified OS which the BSDs maintain themselves.
- jcranmer 7y ago
  > For example, Chromium. Any BSD developer would know that attempting to upstream their port there is dead in the water. AFAICT, They Chromium devs only care about Win, Mac, Linux and nothing else. Likewise for Firefox, which is why the BSDs don't get official releases. Firefox does maintain ports for the BSDs. https://bugzilla.mozilla.org/show_bug.cgi?id=1598511 https://bugzilla.mozilla.org/show_bug.cgi?id=1598511, an OpenBSD-only issue, was fixed less than a month ago. The BSDs qualify as "Tier 3" in Mozilla's build terminology, which means that the onus is on external contributors to identify problems and propose fixes, as there is no continuous integration support for these architectures.
  - clarry 7y ago
    > The BSDs qualify as "Tier 3" in Mozilla's build terminology, which means that the onus is on external contributors to identify problems and propose fixes, as there is no continuous integration support for these architectures. In other words, Firefox doesn't maintain the port for OpenBSD, OpenBSD developers do.
- inferiorhuman 7y ago
  By extension this also renders things like Electron unsupported on the BSDs.
ubercow13 7y ago
Is the shared library issue actually related to this? In the OP they are talking about a Rust upgrade which is presumably only a build-time dependency.
cptskippy 7y ago
So this is kind of like the DLL hell Windows had?
- gdxhyrd 7y ago
  Kind of, yes. Nowadays (for almost 20 years actually), Windows supports and encourages programs to deploy their dependencies in their own folder. It is trivial to do for developers and works perfectly for users. Some people claim security updates are a problem, but many apps do not deal with untrusted input for starters. For those that do, like browsers, servers or your office suite, you really should be using one that is supported and updated automatically.
Ericson2314 7y ago
I have been interested in finding BSD users who are interested in https://nixos.org/nix/ https://nixos.org/nix/ or even a NixOS/kBSD. It solves all these problems and in my view is the continuation of the spirit of the port system. (The problem with bundling isn't disk space, but composition. Individual applications can compose fine, but libraries can't if they link other libraries at different versions, and use those library's types (ABI) in their own interface (ABI). To solve this problem you need to distinguish between public vs private dependencies in your package manager.)
- yjftsjthsd-h 7y ago
  > a NixOS/kBSD Oh good, I'm not the only one thinking about this:) My last idea was basically "run nix on top of a BSD (replacing the package manager), then make nix package(s) for the base system". Which sounds doable, actually.
- floatboth 7y ago
  Join #freebsd-nix on freenode. It's quiet here, but it exists.
bscphil 7y ago
> I think the benefits of saving disk space by sharing libraries do not justify the inconveniences we, desktop BSD and Linux users, obtain from being forced to disrupt and upgrade hundreds of installed software packages, just because one single desired upgrade requires to pull its dependencies It's funny, from your point of view having a centralized repository with a (usually) single (usually) latest version of a library is a bad thing that may be (and probably isn't) justified by the goal of saving space. From my point of view having a centralized repository with a (usually) single (usually) latest version of a library is an awesome thing that I would leave any other ecosystem to get, and the space savings is just a bonus that doesn't much matter. Most dependency maintainers don't provide updates for more than a few versions of their software. When one piece of software depends on -latest and another piece of software depends on -legacy, you can ship both with the central repository model. In the Linux distributions I've used this is a solved problem. Arch Linux has five different versions of the JRE that are separately installable.
- celticmusic 7y ago
  I left arch linux because I got tired of all the breakages, so not a great example.
  - bscphil 7y ago
    Unless Arch Linux broke for you because they ship multiple versions of some packages, that's not really relevant. This is a thing all the popular Linux distributions do as far as I know. Also, personally I've stuck with Arch Linux for 6-7 years now because it's the first distribution I found that consistently works for me, so I don't know that anecdotes will get us very far either way.
  - chmln 7y ago
    This has to be one of the most inaccurate memes of all time. To counter with anecdata: I've been running Arch Linux testing repositories for over a year now with no issues whatsoever.
    - 7y ago
haroldp 7y ago
Would it be possible to run Firefox in it's own jail? All of my FreeBSD boxen are servers, so I don't have any experience with this but... put your crustier apps in containers so they don't poop in your main sandbox. Works for servers
- dddddaviddddd 7y ago
  Yes. Example specifically for FF on FreeBSD: https://wiki.freebsd.org/VladimirKrstulja/Guides/JailingGUIApplications https://wiki.freebsd.org/VladimirKrstulja/Guides/JailingGUIA...
jmclnx 7y ago
It is not disk space you should be concerned with saving, but real memory usage. I wonder how much memory a statically liked Firefox uses under a heavy load. I checked with esr (what I use) and it is about a bit more that 1.3G excluding shared. So as firefox creates threads, I would think memory could get tight with a statically linked FF. I fully agree with what OpenBSD has decided, I think the only thing worse that compiling Firefox is Gnome 3 :)
- ComputerGuru 7y ago
  Statically linked processes can’t share dependency pages with other, different processes. Multiple instances of the same process or multiple threads don’t have to incur that same penalty.
- ENOTTY 7y ago
  Seems to me that more aggressive memory deduplication at the kernel or hardware level should be able to mitigate memory explosion from using only statically linked binaries or letting “big” frequent updated projects manage their own dependencies.
- gdxhyrd 7y ago
  > So as firefox creates threads, I would think memory could get tight with a statically linked FF. Threads have nothing to do with that and don't increase memory usage due to dependencies.
djsumdog 7y ago
Is that what's going on? I wonder how the Gentoo maintainers are dealing. They frequently unbundle libraries, but I've also see a lot of packages with use flags like system-jpeg or system-sdl to force the ebuild to use a system library instead of the built in. Firefox, Libreoffice and others big applications typically take a long time to compile because they have so much stuff built-in instead of depending on the system libs. They gain a lot of stability and predictability but you can have dependent libs with security issues too.
- joecool1029 7y ago
  > I wonder how the Gentoo maintainers are dealing. I spoke to the gentoo firefox package maintainer last night. He even has firefox building on aarch64 with musl, so things are pretty good. For more common targets, there's also the option of using the official binaries from Mozilla. Gentoo policy is to attempt to provide a way to use system libraries where possible except when those libraries are too heavily modified. If something breaks, they use the bundled package. You can see that's currently happening with harfbuzz in firefox until they bringup the patch: https://github.com/gentoo/gentoo/blob/master/www-client/firefox/firefox-72.0.1.ebuild https://github.com/gentoo/gentoo/blob/master/www-client/fire... From experience, it's usually fine to use system libs when available unless those system libs are unstable or development releases. Then all bets are off.
zajio1am 7y ago
This seems to me like a false dillema between forced upgrade due to dependencies and not using shared libraries. I think that there is in general no technical problem to have multiple versions of the same library installed (files in /usr/lib are versioned anyway and for other data the libraries can be compiled with prefix including version). Then, package system could handle different major/minor versions of libraries as different entities, handle upgrade of major/minor versions through dependencies from installed/upgraded packages, and only upgrade library directly for patch-version change (i.e. security bugfix). In such setup, there would be several instances of a popular library installed on a system, but not say hundreds partial instances statically compiled-in to applications. And it still allows simple updates of libraries for patch fixes.
jrockway 7y ago
I agree. I also think that large projects should just vendor their dependencies, including compilers, so you git clone the thing, type "bazel build", and have a working binary. (I am fine if the dependencies are not checked in to the repository, a bazel WORKSPACE file is fine with me. It records a checksum for all dependencies, so even if you reach out to the Internet to get it, you get the same bytes as the developers working on the code.) Building and distributing software should be very simple. But it's not, because of an accumulation of legacy tools (shared libraries) and practices ("please install these 6000 dependencies to build our project").
- strictfp 7y ago
  Sure, that gives you working binaries. But also security problems, because you don't control which versions of libraries you are using. Suddenly, all individual software packages must do their own security updates to keep the system secure. So the whole thing is a balance.
  - blackhaz 7y ago
    Yes. But if we can extract versions of libraries used to build packages then it would be easy to audit the system by cross-checking against CVEs. Those who prioritize security would remove the affected package(s) until a new version is available. My point here is to be able to install a new package when it's out, without disrupting the whole environment. For FreeBSD, for example, the new Firefox is already available. I have installed it, and it wouldn't run. I had to auto-update 300 MB of other stuff, including LibreOffice, PyCharm and even TeXLive, to get the system up to date for the new Firefox to be able to run.
  - jrockway 7y ago
    You do control what version of libraries you're using. You include the exact SHA256 of every dependency of every dependency, down to the toolchain itself. If you're saying "your distribution can't automatically update you if libc is vulnerable to something", that's true. More CPU time is required to react to major vulnerabilities, as everything has to be recompiled. However, it's not much CPU time, and the downsides of requiring more compute time are lower than the upsides of knowing exactly where your dependencies come from. And having your "getting started" instructions be "1) install bazel 2) bazel run //your:binary".
pmontra 7y ago
I never experienced that catastrophic breakage and I'm using Ubuntu as my only desktop OS since January 2009. When did that happen to you? I have no problems with the standard approach. Actually, it makes me feel safe: a maintainer fixes a vulnerability in a shared library and all apps are fixed. If all apps shipped their own libraries I'd have to download a new version of all apps, if they ever care to fix their code.
ihucos 7y ago
Generally it could be possible to run single (bloated) applications inside a container. Here is the solution I wrote for Linux: http://zpkg.plash.io/ http://zpkg.plash.io/
- Mirioron 7y ago
  I think containerization is going to be the future for applications in general. It would certainly make using different versions of Python easier.
  - gdxhyrd 7y ago
    Why? Using different Python versions is a solved problem and no containers are required for that.
  - stock_toaster 7y ago
    Is containerization any better than static binaries for compiled apps (not talking about python/ruby/js apps here, but instead compiled ones like firefox)?
    - sitkack 7y ago
      Many applications were never designed to be compiled entirely static, as much as I’d like that to be the case it isn’t possible. Containerization satisfies the contract the application expects. That is really the problem it fixes, dependency hell and dynamic linking.
      - Wowfunhappy 7y ago
        That’s great on a personal level, but bigger picture, aren’t we going about this the wrong way? If most people are using containers because “applications were never designed to be compiled entirely static”, developers should start designing their applications so they can be compiled entirely static.
        7y ago
zokier 7y ago
Isn't that the whole point of "stable" distros like Debian and RHEL that you can run apt upgrade/yum update and it'll never break your system? I thought that's the big reason they bend over backwards to backport patches and muck around packages so that they retain compatibility.
- blackhaz 7y ago
  Perhaps, but in which case this concept actually adds value? Imagine you have a working environment (development, scientific research, etc) carefully set up to do your job. You wish to upgrade, say, your web browser, but now have to upgrade the whole environment. Changing versions can be very disruptive in many situations.
  - zokier 7y ago
    > Changing versions can be very disruptive in many situations. You assert that, but is that really true? Updating "stable" distro should be non-event, the very opposite of disruptive.
- ownagefool 7y ago
  Yep. Imagine you're rolling out *nix 5000 endpoints; change is going to make the problem hard, so the idea is stables ABI/APIs should be the same, so it can like a platform you can build against. The problem is, browsers pretty much have to be the latest versions, or websites won't work, and most hardware improvements are going to need a new kernel (which to be fair, given linus don't break user space position, is a probably safer than updating userspace), so it doesn't necessarily work for all use cases. A common approach in webdev has been to keep current, but run a test suite so you know what breaks, and deal with it when it happens, but I don't think the average infrastructure person has kept up with web dev here, and unless you're running containers, you're probably still on a stable OS.
lmilcin 7y ago
If you are praying every time you are updating your distro you might be wanting to switch to Linux. I heard it can tolerate two different applications using different versions of the same dependency...
nils-m-holm 7y ago
> The whole ecosystem falls down like a card house [...] > praying my Python and other projects survive [...] And this is exactly why I never update. Every 10 years or so I go through the pain of installing a new version of the OS, mostly because the browser no longer works on most websites. But then, now that the web becomes more and more uninteresting to me, I might as well keep the current version until the hardware breaks down. Security? Sure, but not at the cost of ruining half of my installed software on a regular basis! If there was a statically linked version of BSD, I would switch to it in the blink of an eye.
- Wowfunhappy 7y ago
  What type of system is this? I appreciate the sentiment but I’m surprised you manage to pull it off seemingly easily. You don’t run into software compatibility issues? (I’m trying to do something similar by downgrading to an old version of macOS more-or-less permanently, because I don’t like where the platform has gone.)
  - nils-m-holm 7y ago
    I am running various flavors of BSD, mostly FreeBSD and OpenBSD. The only compatibility issues are with web sites that require the latest browsers.
enriquto 7y ago
> I think the benefits of saving disk space by sharing libraries do not justify the inconveniences we This is really sad. It seems that we are moving from the complexity of shared libraries to the complexity of "docker images" and the like, where the whole OS is embedded in each program for convenience. The sweet spot of static executables has been shamefully bypassed.
devwastaken 7y ago
The devs that complain about <1gb of space over an important piece of software are living in the early 2000s still. They haven't figured out a terabyte on disk is worth about $30, with ssd terabyte at $100.
diffeomorphism 7y ago
As an end user I view it the exact opposite way. On windows I have dozens of auto updaters running in the background that may or may not fix security issues in any reasonable amount of time. I also have lots of different versions of .net, libraries etc. installed, some with security support some without. Also, an issue in a central library will require hundreds of updates, some of which may never come. In contrast on linux I have exactly one updater and can be certain my software uses the fixed version. I don't care about disk space saving or RAM savings (though those are nice), I care that stuff gets fixed and is secure. If you don't enforce that you trade convenience (for the developers, not the user) for security.

progval 7y ago

> auto-update more than a thousand packages, praying my Python and other projects survive, just because I want a browser update You may want to check out Nix. It's a package manager which isolates each program's dependencies; so you can have multiple versions of the same package. > or a security fix having a centralized repository like this helps in case a it's a library that needs a security fix, because you only need a single update to apply the fix to all applications. Applications that bundle their own deps (including Docker images) will need to publish their own update with the updated version of the lib.

l0b0 7y ago
> You may want to check out Nix. Nix is a fantastic concept, and I hope it takes over the world. But the NixOS packages are a mess. I tried it for a few months last year before giving up after several packages and even whole collections of packages became unusable even in the stable repository. The repository needs some serious reworking before Nix can really shine.
- zrm 7y ago
  It's really trading one problem for a different one. The reason traditional distributions only package one version of a given library and make everything use that same version is so they only have to maintain one version of the library. If you allow every application to choose its own version then they all choose a different one, which means someone then has to continue to maintain every different version of the library separately. Doing that isn't much if any of a reduction in workload compared with getting every application to use the version the distribution ships, and is much more likely to end up with a situation where dozens of versions of the same library exist and half of them are broken in some way. The better solution is for libraries to only break compatibility between major versions. Then the packager can ship some suitably recent minor version for each major version of the library, at most two or three major versions are supported at any given time which keeps the maintenance overhead feasible, and every application can successfully link against one of those major versions or another.
  - avar 7y ago
    It doesn't entail any of that. Traditional distributions like Debian allow you to install any number of library combinations, they just do so badly by requiring you to do so on different installations updated at different times. Supporting the presence of multiple library combinations doesn't mean that you "support" all version combinations in the sense of supporting all combinations of their versions being used in the wild. It just means when users upgrade they can do so gradually for some sets of packages at a time. You can emulate this on traditional distributions by having a single VM image you use for your browser, then cloning the image, updating packages, and using that one for your E-Mail etc. Having a mechanism like this should mean less support is needed from the distribution, because system-wide breakages are less likely to occur. You'll get the same bugs, but users can easily back out say an OpenSSL version with a security fix for the 5% of packages it breaks with, while retaining the fix for their browser & other high-risk packages. For example, I currently can't upgrade my firefox version on Debian testing because it ">=" depends on a version of a couple of libraries that some 100-200 other packages on my system don't like. In Debian testing this sort of thing happens occasionally and will get fixed sooner than later, but that it happens at all is just an emergent property of how the versions are managed. If I were to run the latest firefox with those new versions and not touch the other packages until they're recompiled both myself and the package maintainers would be exposed to fewer bugs, not less. I'd get a bugfixed firefox today, and they wouldn't need to deal with bug reports about a broken upgrade, or firefox bugs in an older version users like me can't upgrade past simply because firefox happens to share the use of a popular OS library.
    - zrm 7y ago
      > Supporting the presence of multiple library combinations doesn't mean that you "support" all version combinations in the sense of supporting all combinations of their versions being used in the wild. It's not about combinations. It's about, there's a serious bug in the library and now there are 79 different actively used versions of it that need to be patched instead of 2, which means half of them never get fixed. > You'll get the same bugs, but users can easily back out say an OpenSSL version with a security fix for the 5% of packages it breaks with, while retaining the fix for their browser & other high-risk packages. You'll get the same bugs and then the 5% of packages a security update breaks are broken until they're fixed, which should happen promptly because broken packages should be very high priority. Is it really better in that short time for the other 5% of applications to "work" but be insecure and get your system compromised because the new vulnerability is being actively exploited? The real solution is for sufficient testing to be done during the embargo period so that by the time the new version goes wide it doesn't actually break anything. > For example, I currently can't upgrade my firefox version on Debian testing because it ">=" depends on a version of a couple of libraries that some 100-200 other packages on my system don't like. Right, that's a problem. It's valid to need version >= 5.6 of some library, but something has gone wrong if some other package specifically needs version 5.5 -- version 5.6 should be able to satisfy that dependency. Or if it can't then "5.6" should be 6.0 and it should be possible to have 5.x and 6.x installed together, but that should happen only rarely so that at most two or three mutually incompatible versions are in active use at once. In general the latest version should be able to satisfy applications expecting older versions and then you need only a suitably recent version and nothing else, and the lack of that is the real issue.
- millettjon 7y ago
  I've been using it for 2 years with very few problems.
danieldk 7y ago
You may want to check out Nix. It's a package manager which isolates each program's dependencies; so you can have multiple versions of the same package. In principle true, yes. But it comes with a cost. Even the nixpkgs Firefox maintainers are considering to only ship Firefox ESR starting with NixOS 20.03: https://discourse.nixos.org/t/firefox-on-19-09-could-be-marked-as-insecure-or-downgraded-to-esr-68/4577 https://discourse.nixos.org/t/firefox-on-19-09-could-be-mark... https://github.com/NixOS/nixpkgs/pull/72126 https://github.com/NixOS/nixpkgs/pull/72126
- progval 7y ago
  I won't blame them. Firefox is a huge piece of software, and is updating every month. It's a bit shitty of Mozilla to expect every single distribution (most of which are run by volunteers) to package all their new dependencies and check everything works every single month.
  - rusk 7y ago
    This sounds like ”move fast and break things” writ large, and a salutary lesson in the flaws of this approach.
  - jcelerier 7y ago
    but you can just use Mozilla's official binary packages, which are tested to work with a specific set of libraries, and I think are built with higher optimization levels (O3/pgo) than what distros conservatively do (O2) because of cargo-culting early-2000 gentoo forums (that was the case a few years ago, not 100% sure it is still the case today as I could not find easily the current release flags).
    - progval 7y ago
      Mozilla doesn't provide binary packages for all architectures and OSs supported by Nix/NixOS Maintainers also sometimes need to patch software to conform to their distribution's norm. For example, Debian patches it to accept extensions installed with APT. And I'm glad distribution maintainers always recompile software themselves, it helps making sure: 1. binaries are really made from the open source code (I know, it's not bullet-proof) 2. it's actually humanly possible to actually compile it from source (my pet peeve in this area is Kafka: it's used everywhere, but no one seems to know how to actually fully compile it from sources anymore)
jrockway 7y ago
Is this kind of like Ubuntu's "snap" thing? (I know snap involves cgroups, which aren't strictly necessary to give Firefox a different version of rust than everything else... but it seems widely deployed and easy to use, at least if you're on Ubuntu.)
xvilka 7y ago
Guix[1] also. It's written in Scheme but also solves the bootstrapping problem with minimal seed[2]. [1] http://guix.gnu.org/ http://guix.gnu.org/ [2] https://guix.gnu.org/manual/en/html_node/Reduced-Binary-Seed-Bootstrap.html https://guix.gnu.org/manual/en/html_node/Reduced-Binary-Seed...

Avamander 7y ago

> praying my Python and other projects survive, just because I want a browser update or a security fix. You mean you aren't familiar with Python's virtual environment system exactly intended for isolating development dependencies from system ones but you're blaming the distribution. Please.

sunaurus 7y ago
None of my virtualenvs (on Linux or on OS X) survived the 3.7 -> 3.8 upgrade. Are you saying they shouldn't have broken?
- pmontra 7y ago
  Virtualenv lets you use only a Python interpreter that's already installed on your system. Usually the choice is between the latest v2 and v3 interpreter, at least on Debian derivatives. Every virtualenv is autoupgraded each time the OS upgrades its Pythons. If you want to use multiple versions of Python and pin them, you should use something like asdf [1] and its Python plugin [2]. It's a version manager with support for many languages and more. I used it for Elixir, Node, PHP, Ruby and even PostgreSQL (I need different database versions in different projects.) I never used it for Python because I'm OK with being on the latest version so far. With asdf I would expect to do something like this asdf install python 3.7.5 mkdir my375project cd my375project asdf local python 3.7.5 # pick the version to use here python3 -m venv py375 --python=~/.asdf/installs/python/3.7.5/bin/python ~/.virtualenvs/py375/bin/activate This virtualenv is going to stay on 3.7.5 forever. [1] https://github.com/asdf-vm/asdf https://github.com/asdf-vm/asdf [2] https://github.com/danhper/asdf-python https://github.com/danhper/asdf-python
rleigh 7y ago
Virtualenvs don't protect you completely. If any of the modules in the virtualenv depend upon libraries on the host system, and a lot of Python modules do either directly or indirectly, then there is a chance of breakage if any of the ABIs of these modules break or there's an incompatible ABI between any of the modules. Even virtualenvs themselves can cause the breakage. Take "pylibtiff". It embeds a copy of libtiff. But, it's old and incompatible with other libraries on the base system. This can be libraries it depends upon, or it could be another copy of the same library. Either way, virtualenvs can cause more problems than they solve.

smartmic 7y ago

Hm. For me this is a question about interface design (vs. implementation). IMHO Unix programming philosophy is a lighthouse how to solve this commendable. In an ideal world you should have to upgrade base program and dependent libraries only if their interfaces have changed. EDIT: And of course for security reasons - but then you only upgrade the affected player and not huge parts of your installed base. What I want to say - this is all about the ruling design philosophy.

floatingatoll 7y ago

RedHat’s “streams” model will certainly do a much better job of handling this than the other distributions do today. I hope that the need for having multiple parallel versions of a dependency coexist is incorporated into the other distros, because I’ve lost a lot of sanity this past two decades to the assumption that “one installed version should be enough for anybody” on Linux and BSD servers.

ethbro 7y ago
FYI, for those like me whose last memory of RedHat packaging was pre-yum: https://developers.redhat.com/blog/2018/11/15/rhel8-introducing-appstreams/ https://developers.redhat.com/blog/2018/11/15/rhel8-introduc...
offmycloud 7y ago
I hope so, but Fedora is dragging their feet putting out a patched Firefox package on Fedora 30, which is still suppoted.
- cesarb 7y ago
  You can check the status for the firefox package in Fedora at https://admin.fedoraproject.org/updates/ https://admin.fedoraproject.org/updates/ which shows firefox-72.0.1-1.fc30 as "testing" (that is, you can install through "yum --enablerepo=updates-testing update firefox"). The reason for it not being promoted to stable yet is, according to that page, "disabling automatic push to stable due to negative karma" - that is, the update was marked as broken by a couple of people.
seanparsons 7y ago
Already handled in NixOS by Nix.
Qerub 7y ago
AppStream does not allow installation of multiple versions of the same app AFAIK. I believe this is what they refer to in the clumsy sentence "The one disadvantage of Application Streams from SCLs is that no two streams can be installed at the same time in to the same userspace. However, in our experience, this is not a common use case and is better served using containerization or virtualization to provide a second userspace." in the linked article in the sibling comment.

markstos 7y ago

Complex apps can bundle their own dependencies-- that's what the Flatpak and Snap package formats do. You can also run Firefox in a Docker or LXD container with it's dependencies. By sharing the X11 or Wayland socket with the docker, the apps can appear on your main desktop.

jackhalford 7y ago
none of those solutions apply the BSDs though
- LeoPanthera 7y ago
  Can you run graphical applications inside a BSD jail?
  - napsterbr 7y ago
    You can with SSH and X11 forwarding. Whether that's practical is a different question and may vary according to the application and user
    - toast0 7y ago
      You wouldn't need to use SSH, just linking/bind mounting the X socket and xauth into the jail (plus/minus whatever shm stuff).
  - beefhash 7y ago
    Pedantic note: Jails are not a BSD concept, they're a FreeBSD concept. OpenBSD has no notion of jails.
  - dddddaviddddd 7y ago
    Yes. Here's an example for Firefox on FreeBSD: https://wiki.freebsd.org/VladimirKrstulja/Guides/JailingGUIApplications https://wiki.freebsd.org/VladimirKrstulja/Guides/JailingGUIA...
- 7y ago

ChickeNES 7y ago

I wish NeXTsteps's app bundles had caught on outside of OSX.

hoistbypetard 7y ago
Me too. Especially as combined with fat binaries in the OS X rosetta years. It'd make it a lot easier to package something that would run on arm, arm64 and amd64 in a more accessible way for users.
stock_toaster 7y ago
AppImage[1] kind of works that way. I'm surprised it hasn't caught on more, tbh. I don't think it supports multiple architectures like app bundles can (fat bundles) though. [1]: https://appimage.org https://appimage.org
- lokedhs 7y ago
  One of my applications is distributed through Appimage. It works really well. I really feel it should support containerisation in a similar way as Flatpak does. In my opinion, the idea that any application on your system can can touch all your files and access all your other applications is really problematic. You're always just one broken (intentionally or not) application form having your entire system compromised. As best as I can tell, the only solutions that even tries to address this is Qubes OS and Flatpak.
pjmlp 7y ago
On Windows they are called xcopy install, and they have also caught on mobile OSes.

sandov 7y ago

I'll go one step further: The whole "centralized, trusted repository that has all your apps" system is wrong at a fundamental level. The way shared libraries are used in Linux is built upon the assumption that package managers and centralized repositories are the right way to do things.

Analemma_ 7y ago
I agree, and I think that people are too emotionally invested in the package manager concept to back out now. I mean, for years Linux proponents have been touting it as the key advantage over software distribution on Macs/PCs.
- hoistbypetard 7y ago
  The existence and popularity of homebrew, macports and fink (on mac) and ninite, chocolatey, cygwin, mingw and nuget (on windows) would suggest that the concept appeals to mac and windows users also.
- microcolonel 7y ago
  I mean, it is a key advantage, on ArchLinux. If you don't have a rolling release, it's pure pain: you're often just stuck with old versions of software which aren't any more stable than the current stable release.
  - TheCraiggers 7y ago
    It's only pure pain if you want bleeding edge AND stability. Non-rolling releases are just fine if you only need new features once a year.
    - snazz 7y ago
      Non-rolling releases might work with smaller software where your distro maintainers can backport security bugfixes easily, but as soon as your software gets too big and complicated the distro maintainers won't be able to backport fixes themselves and will have to just resort to packaging the latest version of certain packages (like Debian had to do with Chromium and Firefox). Also, for the most part, the "Arch is unstable" thing hasn't been true for a long time. I find that it's far more stable than Debian or Ubuntu LTS on my machine because of newer kernels containing newer hardware drivers.

zozbot234 7y ago

Outside packages should NOT be disrupted given semver. The whole point of using shared objects (dynamically-linked libraries) is so that when a problem arises you can update whatever pieces of code in a centralized, system-wide store and every single one of the projects you use can benefit from the new, up-to-date version. Using the latest version is just the right thing to do.

stanferder 7y ago
Does it actually work though? It seems like there is a significant difference between the expectation that semantic versioning should result in no disruption, and the experience of the parent to your comment. I'd be interested to know exactly what went wrong that lead that poster to be pessimistic.
- jiveturkey 7y ago
  Yes it does actually work and has for decades. The poster is either confused or doing something different, I expect. The specific notice mentions rust dependencies. Rust does not have shared libraries, so a Rust [security] update means all rust binaries must be completely rebuilt. That seems to be part of OpenBSD's concern, and perhaps this has "triggered" the poster. user blackhaz random forum post found: > mariourk, you're definitely not alone. The same breakage happens to FreeBSD desktops as well. I have been vocal around the forum a few times about this. Not every desktop user wants to upgrade all their packages every X months. The way the default repositories are structured, the upgrades are forced onto users. I am a huge proponent of using application bundles, like on Mac OS, as quite often those upgrades break complex desktop apps too. I don't mind upgrades but sometimes I need to stick with a specific version of software, or roll back after something went wrong, and if it came as a bundle with its own dependencies, it wouldn't have to depend on other stuff that is being "force-upgraded" periodically. So his complaint is about the way FBSD handles updates; he is generally incorrect about shared libraries; and his comment is completely OT for this thread.
  - zozbot234 7y ago
    > Rust does not have shared libraries, so a Rust [security] update means all rust binaries must be completely rebuilt. What's the concern with that, exactly? OpenBSD is a security-focused distribution, do they not do reproducible builds?

NelsonMinar 7y ago

Dumb solution with a blunt hammer; why not run Firefox in a container like Docker?

pferde 7y ago
I do something like that. I have a separate minimal chroot directory with just Firefox installed and configured, and use systemd-nspawn to start a container. The container is using a snapshot (btrfs) of the chroot directory, which gets deleted when I close the Firefox instance. I use this not as my main browser, but only when I can't avoid visiting "the wild web", meaning unknown or untrusted sites, or even sites that use too much javascript for me to feel safe allowing in my main browser.

rvz 7y ago

Well the reality is, some developers of these complex software complain how their software ported to alternative OSes is difficult to maintain even when the BSD devs would offer to maintain it, which is why the BSDs are always on their own here in terms of porting, testing, packaging and updating the software in question. For example, Chromium. Any BSD developer would know that attempting to upstream their port there is dead in the water. AFAICT, They Chromium devs only care about Win, Mac, Linux and nothing else. Likewise for Firefox, which is why the BSDs don't get official releases. I feel some sympathy for OSes like the BSDs that for "some" software they are limited by the arrogance of other open-source maintainers who would rather go for maintaining the convoluted distros and testing with a huge disintegrated software-stack than testing a single unified OS which the BSDs maintain themselves.

jcranmer 7y ago
> For example, Chromium. Any BSD developer would know that attempting to upstream their port there is dead in the water. AFAICT, They Chromium devs only care about Win, Mac, Linux and nothing else. Likewise for Firefox, which is why the BSDs don't get official releases. Firefox does maintain ports for the BSDs. https://bugzilla.mozilla.org/show_bug.cgi?id=1598511 https://bugzilla.mozilla.org/show_bug.cgi?id=1598511, an OpenBSD-only issue, was fixed less than a month ago. The BSDs qualify as "Tier 3" in Mozilla's build terminology, which means that the onus is on external contributors to identify problems and propose fixes, as there is no continuous integration support for these architectures.
- clarry 7y ago
  > The BSDs qualify as "Tier 3" in Mozilla's build terminology, which means that the onus is on external contributors to identify problems and propose fixes, as there is no continuous integration support for these architectures. In other words, Firefox doesn't maintain the port for OpenBSD, OpenBSD developers do.
inferiorhuman 7y ago
By extension this also renders things like Electron unsupported on the BSDs.

ubercow13 7y ago

Is the shared library issue actually related to this? In the OP they are talking about a Rust upgrade which is presumably only a build-time dependency.

cptskippy 7y ago

So this is kind of like the DLL hell Windows had?

gdxhyrd 7y ago
Kind of, yes. Nowadays (for almost 20 years actually), Windows supports and encourages programs to deploy their dependencies in their own folder. It is trivial to do for developers and works perfectly for users. Some people claim security updates are a problem, but many apps do not deal with untrusted input for starters. For those that do, like browsers, servers or your office suite, you really should be using one that is supported and updated automatically.

Ericson2314 7y ago

I have been interested in finding BSD users who are interested in https://nixos.org/nix/ https://nixos.org/nix/ or even a NixOS/kBSD. It solves all these problems and in my view is the continuation of the spirit of the port system. (The problem with bundling isn't disk space, but composition. Individual applications can compose fine, but libraries can't if they link other libraries at different versions, and use those library's types (ABI) in their own interface (ABI). To solve this problem you need to distinguish between public vs private dependencies in your package manager.)

yjftsjthsd-h 7y ago
> a NixOS/kBSD Oh good, I'm not the only one thinking about this:) My last idea was basically "run nix on top of a BSD (replacing the package manager), then make nix package(s) for the base system". Which sounds doable, actually.
floatboth 7y ago
Join #freebsd-nix on freenode. It's quiet here, but it exists.

bscphil 7y ago

> I think the benefits of saving disk space by sharing libraries do not justify the inconveniences we, desktop BSD and Linux users, obtain from being forced to disrupt and upgrade hundreds of installed software packages, just because one single desired upgrade requires to pull its dependencies It's funny, from your point of view having a centralized repository with a (usually) single (usually) latest version of a library is a bad thing that may be (and probably isn't) justified by the goal of saving space. From my point of view having a centralized repository with a (usually) single (usually) latest version of a library is an awesome thing that I would leave any other ecosystem to get, and the space savings is just a bonus that doesn't much matter. Most dependency maintainers don't provide updates for more than a few versions of their software. When one piece of software depends on -latest and another piece of software depends on -legacy, you can ship both with the central repository model. In the Linux distributions I've used this is a solved problem. Arch Linux has five different versions of the JRE that are separately installable.

celticmusic 7y ago
I left arch linux because I got tired of all the breakages, so not a great example.
- bscphil 7y ago
  Unless Arch Linux broke for you because they ship multiple versions of some packages, that's not really relevant. This is a thing all the popular Linux distributions do as far as I know. Also, personally I've stuck with Arch Linux for 6-7 years now because it's the first distribution I found that consistently works for me, so I don't know that anecdotes will get us very far either way.
- chmln 7y ago
  This has to be one of the most inaccurate memes of all time. To counter with anecdata: I've been running Arch Linux testing repositories for over a year now with no issues whatsoever.
  - 7y ago

haroldp 7y ago

Would it be possible to run Firefox in it's own jail? All of my FreeBSD boxen are servers, so I don't have any experience with this but... put your crustier apps in containers so they don't poop in your main sandbox. Works for servers

dddddaviddddd 7y ago
Yes. Example specifically for FF on FreeBSD: https://wiki.freebsd.org/VladimirKrstulja/Guides/JailingGUIApplications https://wiki.freebsd.org/VladimirKrstulja/Guides/JailingGUIA...

jmclnx 7y ago

It is not disk space you should be concerned with saving, but real memory usage. I wonder how much memory a statically liked Firefox uses under a heavy load. I checked with esr (what I use) and it is about a bit more that 1.3G excluding shared. So as firefox creates threads, I would think memory could get tight with a statically linked FF. I fully agree with what OpenBSD has decided, I think the only thing worse that compiling Firefox is Gnome 3 :)

ComputerGuru 7y ago
Statically linked processes can’t share dependency pages with other, different processes. Multiple instances of the same process or multiple threads don’t have to incur that same penalty.
ENOTTY 7y ago
Seems to me that more aggressive memory deduplication at the kernel or hardware level should be able to mitigate memory explosion from using only statically linked binaries or letting “big” frequent updated projects manage their own dependencies.
gdxhyrd 7y ago
> So as firefox creates threads, I would think memory could get tight with a statically linked FF. Threads have nothing to do with that and don't increase memory usage due to dependencies.

djsumdog 7y ago

Is that what's going on? I wonder how the Gentoo maintainers are dealing. They frequently unbundle libraries, but I've also see a lot of packages with use flags like system-jpeg or system-sdl to force the ebuild to use a system library instead of the built in. Firefox, Libreoffice and others big applications typically take a long time to compile because they have so much stuff built-in instead of depending on the system libs. They gain a lot of stability and predictability but you can have dependent libs with security issues too.

joecool1029 7y ago
> I wonder how the Gentoo maintainers are dealing. I spoke to the gentoo firefox package maintainer last night. He even has firefox building on aarch64 with musl, so things are pretty good. For more common targets, there's also the option of using the official binaries from Mozilla. Gentoo policy is to attempt to provide a way to use system libraries where possible except when those libraries are too heavily modified. If something breaks, they use the bundled package. You can see that's currently happening with harfbuzz in firefox until they bringup the patch: https://github.com/gentoo/gentoo/blob/master/www-client/firefox/firefox-72.0.1.ebuild https://github.com/gentoo/gentoo/blob/master/www-client/fire... From experience, it's usually fine to use system libs when available unless those system libs are unstable or development releases. Then all bets are off.

zajio1am 7y ago

This seems to me like a false dillema between forced upgrade due to dependencies and not using shared libraries. I think that there is in general no technical problem to have multiple versions of the same library installed (files in /usr/lib are versioned anyway and for other data the libraries can be compiled with prefix including version). Then, package system could handle different major/minor versions of libraries as different entities, handle upgrade of major/minor versions through dependencies from installed/upgraded packages, and only upgrade library directly for patch-version change (i.e. security bugfix). In such setup, there would be several instances of a popular library installed on a system, but not say hundreds partial instances statically compiled-in to applications. And it still allows simple updates of libraries for patch fixes.

jrockway 7y ago

I agree. I also think that large projects should just vendor their dependencies, including compilers, so you git clone the thing, type "bazel build", and have a working binary. (I am fine if the dependencies are not checked in to the repository, a bazel WORKSPACE file is fine with me. It records a checksum for all dependencies, so even if you reach out to the Internet to get it, you get the same bytes as the developers working on the code.) Building and distributing software should be very simple. But it's not, because of an accumulation of legacy tools (shared libraries) and practices ("please install these 6000 dependencies to build our project").

strictfp 7y ago
Sure, that gives you working binaries. But also security problems, because you don't control which versions of libraries you are using. Suddenly, all individual software packages must do their own security updates to keep the system secure. So the whole thing is a balance.
- blackhaz 7y ago
  Yes. But if we can extract versions of libraries used to build packages then it would be easy to audit the system by cross-checking against CVEs. Those who prioritize security would remove the affected package(s) until a new version is available. My point here is to be able to install a new package when it's out, without disrupting the whole environment. For FreeBSD, for example, the new Firefox is already available. I have installed it, and it wouldn't run. I had to auto-update 300 MB of other stuff, including LibreOffice, PyCharm and even TeXLive, to get the system up to date for the new Firefox to be able to run.
- jrockway 7y ago
  You do control what version of libraries you're using. You include the exact SHA256 of every dependency of every dependency, down to the toolchain itself. If you're saying "your distribution can't automatically update you if libc is vulnerable to something", that's true. More CPU time is required to react to major vulnerabilities, as everything has to be recompiled. However, it's not much CPU time, and the downsides of requiring more compute time are lower than the upsides of knowing exactly where your dependencies come from. And having your "getting started" instructions be "1) install bazel 2) bazel run //your:binary".

pmontra 7y ago

I never experienced that catastrophic breakage and I'm using Ubuntu as my only desktop OS since January 2009. When did that happen to you? I have no problems with the standard approach. Actually, it makes me feel safe: a maintainer fixes a vulnerability in a shared library and all apps are fixed. If all apps shipped their own libraries I'd have to download a new version of all apps, if they ever care to fix their code.

ihucos 7y ago

Generally it could be possible to run single (bloated) applications inside a container. Here is the solution I wrote for Linux: http://zpkg.plash.io/ http://zpkg.plash.io/

Mirioron 7y ago
I think containerization is going to be the future for applications in general. It would certainly make using different versions of Python easier.
- gdxhyrd 7y ago
  Why? Using different Python versions is a solved problem and no containers are required for that.
- stock_toaster 7y ago
  Is containerization any better than static binaries for compiled apps (not talking about python/ruby/js apps here, but instead compiled ones like firefox)?
  - sitkack 7y ago
    Many applications were never designed to be compiled entirely static, as much as I’d like that to be the case it isn’t possible. Containerization satisfies the contract the application expects. That is really the problem it fixes, dependency hell and dynamic linking.
    - Wowfunhappy 7y ago
      That’s great on a personal level, but bigger picture, aren’t we going about this the wrong way? If most people are using containers because “applications were never designed to be compiled entirely static”, developers should start designing their applications so they can be compiled entirely static.
      - 7y ago

zokier 7y ago

Isn't that the whole point of "stable" distros like Debian and RHEL that you can run apt upgrade/yum update and it'll never break your system? I thought that's the big reason they bend over backwards to backport patches and muck around packages so that they retain compatibility.

blackhaz 7y ago
Perhaps, but in which case this concept actually adds value? Imagine you have a working environment (development, scientific research, etc) carefully set up to do your job. You wish to upgrade, say, your web browser, but now have to upgrade the whole environment. Changing versions can be very disruptive in many situations.
- zokier 7y ago
  > Changing versions can be very disruptive in many situations. You assert that, but is that really true? Updating "stable" distro should be non-event, the very opposite of disruptive.
ownagefool 7y ago
Yep. Imagine you're rolling out *nix 5000 endpoints; change is going to make the problem hard, so the idea is stables ABI/APIs should be the same, so it can like a platform you can build against. The problem is, browsers pretty much have to be the latest versions, or websites won't work, and most hardware improvements are going to need a new kernel (which to be fair, given linus don't break user space position, is a probably safer than updating userspace), so it doesn't necessarily work for all use cases. A common approach in webdev has been to keep current, but run a test suite so you know what breaks, and deal with it when it happens, but I don't think the average infrastructure person has kept up with web dev here, and unless you're running containers, you're probably still on a stable OS.

lmilcin 7y ago

If you are praying every time you are updating your distro you might be wanting to switch to Linux. I heard it can tolerate two different applications using different versions of the same dependency...

nils-m-holm 7y ago

> The whole ecosystem falls down like a card house [...] > praying my Python and other projects survive [...] And this is exactly why I never update. Every 10 years or so I go through the pain of installing a new version of the OS, mostly because the browser no longer works on most websites. But then, now that the web becomes more and more uninteresting to me, I might as well keep the current version until the hardware breaks down. Security? Sure, but not at the cost of ruining half of my installed software on a regular basis! If there was a statically linked version of BSD, I would switch to it in the blink of an eye.

Wowfunhappy 7y ago
What type of system is this? I appreciate the sentiment but I’m surprised you manage to pull it off seemingly easily. You don’t run into software compatibility issues? (I’m trying to do something similar by downgrading to an old version of macOS more-or-less permanently, because I don’t like where the platform has gone.)
- nils-m-holm 7y ago
  I am running various flavors of BSD, mostly FreeBSD and OpenBSD. The only compatibility issues are with web sites that require the latest browsers.

enriquto 7y ago

> I think the benefits of saving disk space by sharing libraries do not justify the inconveniences we This is really sad. It seems that we are moving from the complexity of shared libraries to the complexity of "docker images" and the like, where the whole OS is embedded in each program for convenience. The sweet spot of static executables has been shamefully bypassed.

devwastaken 7y ago

The devs that complain about <1gb of space over an important piece of software are living in the early 2000s still. They haven't figured out a terabyte on disk is worth about $30, with ssd terabyte at $100.

diffeomorphism 7y ago

As an end user I view it the exact opposite way. On windows I have dozens of auto updaters running in the background that may or may not fix security issues in any reasonable amount of time. I also have lots of different versions of .net, libraries etc. installed, some with security support some without. Also, an issue in a central library will require hundreds of updates, some of which may never come. In contrast on linux I have exactly one updater and can be certain my software uses the fixed version. I don't care about disk space saving or RAM savings (though those are nice), I care that stuff gets fixed and is secure. If you don't enforce that you trade convenience (for the developers, not the user) for security.