6 ms·
The new APT 3.0 solver
- amelius 3y agoCan anyone explain why libc versions are so often a problem when installing software? I understand that all libraries in an executable need to use the same version of malloc. But otherwise I don't understand the reason for these clashes. Even malloc can be swapped out for another function with the same functionality. And I also don't understand why libc needs to be updated so frequently, anyway.
- codexon 3y agoThey are a problem because gcc automatically links to the latest version of glibc. As to why they don't add an option to specify an older version? I don't know either and it is rather annoying to have to use docker images of older OSes to target older glibc versions. It's just one of many things that prevents linux from being as popular as windows for desktop users.
- IshKebab 3y agoI think the main reason they don't offer a `--make-my-binary-compatible-with-the-ancient-linux-versions-users-often-have` is that GCC/glibc is a GNU project and the are philosophically against distributing software as binaries. I don't think there's any technical reason why it couldn't be done. To be fair to them though, Mac has the same problem. I worked at a company where we had to keep old Mac machines to produce compatible binaries, and Apple makes it hard to even download old versions of MacOS and Xcode. I guess the difference is MacOS is easy to upgrade so you don't have to support versions from 13 years ago or whatever like you do with glibc.
- codexon 3y ago> I think the main reason they don't offer a `--make-my-binary-compatible-with-the-ancient-linux-versions-users-often-have` is that GCC/glibc is a GNU project and the are philosophically against distributing software as binaries. You don't have to statically compile glibc, gcc just needs an option to tell the compiler to target say, version 2.14 instead of the latest one. The newest glibc has all the older versions in it. That's why you can compile on say ubuntu 14 and have it run on ubuntu 24.
- saurik 3y agoNo like, the point is that the only reason you (and I: I do this all the time, including with my open source software... like: no judgment) want to target some old version of glibc is so you can distribute that binary to people without caring as much about what version of the OS they have; but that would be unnecessary if you just gave them the source code and have them compile their own copy for their system targeting the exact libraries they have.
- codexon 3y agoUnfortunately most people don't want to bother compiling, myself included. I tried gentoo one time and it took 1 hour to compile 5 minutes worth of apt-get on ubuntu.
- fweimer 3y agoOnly the dynamically linked bits, the statically linked startup code and libc_nonshared.a are missing from newer versions. Most programs don't need them (who needs working ELF constructors in the main program?). The libc_nonshared.a bits can be reimplemented from scratch easily enough (but we should switch them over to header-only implementations eventually).
- IshKebab 3y agoYes exactly. That's what the flag would do. But it doesn't exist.
- fweimer 3y agoI used to think that binary compatibility benefits proprietary applications, but I'm not so sure anymore. From a commercial perspective, when we break binary compatibility (not that we want to), it's an opportunity for selling more stuff. Many distributions do periodic mass rebuilds anyway and do not need that much long-term ABI compatibility. Binary compatibility seems mostly for people who compile their own software, but have not automated that and therefore couldn't keep up with updates if there wasn't ABI compatibility.
- IshKebab 3y agoI agree. It's annoying for closed source apps but they generally have the resources to deal with it anyway. E.g. with Questa I can just unzip it and run it. No trouble. It's disproportionately annoying for open source projects who don't want to waste their time dealing with this.
- forrestthewoods 3y ago> They are a problem because gcc automatically links to the latest version of glibc. As to why they don't add an option to specify an older version? Because glibc and ld/lld are badly designed. glibc is stuck in the 80s with awful and unnecessary automagic configure steps. ld/lld expect a full and complete shared library to exist when compiling even though it expects a different shared library to exist in the future. Zig solves the glibc linking issue. You can trivially target any old version for any supported target platform. The only thing you actually need are headers and a thin, implementation free lib that contains stub functions. Unfortunately glibc is not architected to make this trivial. But this is just because glibc is stuck with decades of historic cruft, not because it's actually a hard problem.
- einpoklum 3y ago> awful and unnecessary automagic configure steps Steps taken when? When building glibc? And - what steps? > ld/lld expect a full and complete shared library to exist when compiling ... But ld and lld are linkers... > Zig solves the glibc linking issue. But Zig is a language. Do you mean the Zig standard library? The Zig compiler? > The only thing you actually need are headers and a thin, implementation free lib that contains stub functions. Why do you need stub functions at all, if you're not actually using them?
- deleted 3y ago[deleted]
- ploxiln 3y agoThe zig compiler can compile C and C++, using llvm, and it also packages various libc implementations, including glibc, musl, mingw, and msvc, and more for other OSes. Some people use it as a more convenient golang-like cross-compiler. And this whole combination is a smaller download and install than most other toolchains out there. It just took some inspiration and grunt work, to dissect the necessary (processed) headers and other bits of each libc ... hats of to the zig devs. Random blog post I just found about it: https://ruoyusun.com/2022/02/27/zig-cc.html https://ruoyusun.com/2022/02/27/zig-cc.html
- bregma 3y agoHmm. The MSVCRT.DLL/MSVCRTD.DLL not being binary compatible between releases of Visual Studio is the same thing, except of course you can't even combine some modules compiled for debug with modules compiled without debug in the same executable. The Windows problem has always been so so much worse that pretty much all developers simple resorted to shipping the OS system runtime with every package and it's just expected nowadays. It's where the phrase "DLL hell" originated, after all. Not to say the ABI problem isn't real if you want to combine binary packages from different Linux-based OSes. Plenty of solutions for that have cropped up as well: containers, flatpacks, snaps, the list goes on.
- forrestthewoods 3y ago> The Windows problem has always been so so much worse Hard, hard disagree. The problems are somewhat comparable. But if any platform is more painful it's Linux. Although they're similar if you exclude glibc pain. At least in my personal experience of writing lots of code that needs to run on win/mac/linux/android. > pretty much all developers simple resorted to shipping the OS system runtime with every package Meanwhile Linux developers have resorted to shipping an entire OS via docker to run every program. Because managing Linux environment dependencies is so painful you have to package the whole system. Needing to docker to simply launch a program is so embarrassing. > except of course you can't even combine some modules compiled for debug with modules compiled without debug in the same executable That's not any different on Linux. That has more to do with C++.
- graemep 3y ago> Meanwhile Linux developers have resorted to shipping an entire OS via docker to run every program. > Needing to docker to simply launch a program is so embarrassing. I have never needed docker "just to launch a program". Docker makes it easy to provide multiple containerised copies of an identical environment. Containers are a light alternative to VM images. I assume you find the existence of Windows containers just as embarrassing? https://learn.microsoft.com/en-us/virtualization/windowscontainers/about/ https://learn.microsoft.com/en-us/virtualization/windowscont...
- Sesse__ 3y agoLinking to the latest version of glibc is, in itself, not a problem -- glibc hasn't bumped its soname in ages, it is using symbol versioning instead. So you only get a problem if you use a symbol that doesn't exist in older glibc (i.e., some specific interface that you are using changed). As for using an older version of glibc, _linking_ isn't the problem -- swapping out the header files would be. You can probably install an old version of the header files somewhere else and just -I that directory, but I've never tried. libstdc++ would probably be harder, if you're in C++ land.
- fweimer 3y agoRecent libstdc++ has a _dl_find_object@GLIBC_2.35 dependency, so it's not exactly trivial anymore to link a C++ program against a older, side-installed glibc version because it won't have that symbol. It's possible to work around that (link against a stub that has _dl_find_object@GLIBC_2.35 as a compat symbol, so that libstdc++ isn't rejected), but linking statically is much more difficult because libstc++.a (actually, libgcc_eh.a) does not have the old code anymore that _dl_find_object replaces (once GCC is built against a glibc version that has _dl_find_object). This applies to other libraries as well because there are new(ish) math functions, strlcpy, posix_spawn extensions etc. that seem to be quite widely used already.
- ynik 3y agoI find it's best to treat this as a case of "cross-compilation to old glibc version". That is, you don't want to just link against an old glibc version, you want a full cross-compilation toolchain where the compiler does not pick up headers from `/usr/include` by default (but instead has its own copy of the system headers), and where libstdc++ is also linked against that old glibc version. We used https://crosstool-ng.github.io/ https://crosstool-ng.github.io/ to create such a "cross-compiler". Now it doesn't matter which distribution our developers use, the same source code will always turn into the same binary (reproducible builds, yay!) and those binaries will work on older distributions than the developers are using. This allows us to ship dynamically linked linux executables; our linux customers can just unzip + run, same as our windows customers, no need to mess with docker containers. The downside is that we can't just use a library by `apt install`ing it, everything needs to be built with the cross compilation toolchain.
- zX41ZdbW 3y agoI've made a library named "glibc-compatibility": https://github.com/ClickHouse/ClickHouse/tree/master/base/glibc-compatibility https://github.com/ClickHouse/ClickHouse/tree/master/base/gl... When linking with this library before glibc, the resulting binary will not depend on the new symbol versions. It will run on glibc 2.4 and on systems as old as Ubuntu 8.04 and CentOS 5 even when built on the most modern system.
- thrtythreeforty 3y agoHow does this work?
- threecheese 3y agoIn case parent didn’t see your q: https://www.lightofdawn.org/wiki/wiki.cgi/NewAppsOnOldGlibc https://www.lightofdawn.org/wiki/wiki.cgi/NewAppsOnOldGlibc
- adastra22 3y agoYou are a hero.
- cataphract 3y agoAn easier option is to build against musl, remove the DEPENDS with patchelf, and use that. There are a few incompatibilities with glibc (more on aarch64 than am64), but it's manageable (you might need to patch a few headers in musl and provide missing/incompatible functions). And you'll have a binary that works against musl and glibc both.
- kzrdude 3y agoHere is another interesting tool, this one for patching executables https://github.com/corsix/polyfill-glibc https://github.com/corsix/polyfill-glibc
- superkuh 3y agoThank you! This has solved multiple problems I've had for years.
- LtWorf 3y agochroot has existed for many years.
- sunshowers 3y agoIf you're building Rust, check out cargo zigbuild (yes, zigbuild) which lets you target old glibc.
- fweimer 3y agoYou don't have to update, it's just that developers seem to like to run the latest stuff, and many appear to build production binaries on their laptops (rather than in a controlled build environment, where dependencies are tightly managed, and you could deliberately stick to an older distribution easily enough). The dependency errors indicate real issues because of the way most distributions handle backwards compatibility: you have to build on the oldest version you want to support. Those errors happen if this rule is violated. For glibc-based systems, the effect is amplified because package managers have become quite good at modeling glibc run-time requirements in the package-level dependencies, and mismatches result in install-time dependency errors. Admittedly, I'm biased, but I strongly suspect that if we magically waved away the glibc dependency issues, most applications still wouldn't work because they depend on other distribution components, something that's just not visible today.
- o11c 3y agoBecause: * Developers like to be on the latest-and-greatest distro, and rarely perform builds in a chroot. Sometimes this is sheer apathy; other times it is because backporting library dependencies is annoying. * End-users can't even be assumed to be on the latest LTS. Related, almost nobody understands `rpath` and why it should always be used instead of static linking (assuming normal dynamic linking doesn't work of course). There's some common FUD going around but unless you're a setuid or similar program it's not actually true (and even then, it's only an issue with some uses of rpath).
- cookiengineer 3y agoThe problem is actually: nobody adhering to semantic versioning. Hence the reason why there is a separate API/ABI level embedded into a shared object file, which is pretty much always completely different from the library's package version because they broke compatibility so often. Additionally there's lots of libraries that break their symbols and hash tables on every subminor update. (Usually when they use LLVM, but I'm not saying it's LLVM's fault) For example, harfbuzz and libicu are a regular offender of this, meaning they fuck up every downstream project because the method signatures contain a randomized chunk which changes on every single subminor version.
- braiamp 3y agolibicu is deliberately doing this because one of its purpose is to offer Unicode ordering. If consumers try the new ordering on non-rebuild objects, it could lead to data loss. That's what happened with glibc 2.28. [0] If postgres prefers explicit library versions to prevent data corruption, then icu is doing things correctly and making sure that downstream consumers don't shoot themselves in the foot. [0]: https://wiki.postgresql.org/wiki/Locale_data_changes https://wiki.postgresql.org/wiki/Locale_data_changes
- cookiengineer 3y agoHonestly, the more I was researching about what happened in that glibc version update, the more I thought "self inflicted". If you're using NULL terminated strings everywhere, and not UTF8/16, it's your own fault for doing so. Especially if you are building a database which should always contain a length of bytes or at least a CRC check before each entry's contents in its serialized file format to prevent exactly this from happening. (Let alone the question why is there no schema version that should have prevented this from happening in the first place)
- david_draco 3y agoAfter reading so many GPT news item, I was confused what APT is, no explanation on the page, no link ... took me a while to realise it is about debian linux' packaging tool apt.
- lloydatkinson 3y agoThat’s like saying you are confused because an article said Linux and it didn’t explain what it was…
- yjftsjthsd-h 3y ago> The most striking difference to the classic APT solver is that solver3 always keeps manually installed packages around, it never offers to remove them. Y'know, when you put it like that it does seem rather obvious, doesn't it? Principle of least surprise and all that. Edit: To be clear, not knocking anyone for it; hindsight is 20/20 and I've never written a dependency solving algorithm so I'm in no position to judge.
- giancarlostoro 3y agoI love when this happens when I find a new product, I think to myself would be even better if they did this really simple tweak. Then a week or two later they do the same exact change. Apt probably moves a little slower on “big” changes, since theres many implications. I cant even imagine the number of obscure scripts that something like this breaks because the output and behavior is slightly off somehow.
- layer8 3y agoIt’s not that clearcut, because sometimes you forget all the packages you installed manually over the years and aren’t using anymore, and it can make sense to be offered to remove one when it helps resolving a conflict with a newer package that you actually want to use now.
- yjftsjthsd-h 3y agoYeah, I've hit that exact situation on one of my machines running Void Linux (so obviously completely different package manager). I personally think it's best to tell the user that it can't make the given constraints work and suggest which package(s) are the problem, and then if the user is willing to part with them then they can `apt remove foo && apt upgrade` or w/e (which is more or less my experience of how Void's xbps does it).
- inopinatus 3y agoThe healthy choice is to rebuild rather than upgrade any long-lived OS installation (e.g. your desktop) on distro release. Corollary: make it easy on yourself: mount /home separately (and maybe /var too), and keep /etc in scm.
- gsich 3y agoFrom the guy that brought you the keepassxc downgrade. https://github.com/keepassxreboot/keepassxc/issues/10725 https://github.com/keepassxreboot/keepassxc/issues/10725
- k8sToGo 3y agoWhat downgrade are you referring to?
- bjoli 3y agoI never thought I would say this but: I am a pretty recent flatpak convert. This thing just made me more convinced about how it is a good thing to let the Devs deliver the apps themselves and let the distro people do the distro stuff. After accepting flatpak I started using fedora silverblue and then switched to opensuse aeon and I have been very happy. The only pain point was getting Emacs working properly.
- j1elo 3y agoWait so this guy decided to fork the project, and seemingly abused his position to supplant the previous version with his own opinionated fork? All this while disregarding the opinion of upstream devs themselves and being arrogant and stubborn in his replies. What an spectacular way to break things for end users. If there's one thing to learn and apply from Linus, IMHO, is his attitude about NEVER breaking userspace in the Kernel. This lesson can be adapted to most software, and we really should strive to more of it, not less (obviously adjusting to each case; here, replace "userspace" with "user setups")
- westurner 3y ago> Solver3 is a fully backtracking dependency solving algorithm that defers choices to as late as possible. It starts with an empty set of packages, then adds the manually installed packages, and then installs packages automatically as necessary to satisfy the dependencies. [...] > If you have studied SAT solver design, you’ll find that essentially this is a DPLL solver without pure literal elimination DPLL algorithm: https://en.wikipedia.org/wiki/DPLL_algorithm https://en.wikipedia.org/wiki/DPLL_algorithm
- westurner 3y agoprefix-dev/pixi is a successor to Mamba and Conda which wraps libsolv like dnf. https://github.com/prefix-dev/pixi https://github.com/prefix-dev/pixi OpenSUSE/libsolv: https://github.com/openSUSE/libsolv https://github.com/openSUSE/libsolv OpenSUSE/zypper may have wrapped or may still wrap libsolv? TIL libsolv also supports arch .apk and .deb; there's a deb2solv utility to create libsolv .solv files: https://github.com/openSUSE/libsolv/blob/master/tools/deb2solv.c https://github.com/openSUSE/libsolv/blob/master/tools/deb2so... "[mamba-org/rattler:] A new memory-safe SAT solver for package management in Rust (port of libsolv)" (2024) https://prefix.dev/blog/the_new_rattler_resolver https://prefix.dev/blog/the_new_rattler_resolver https://news.ycombinator.com/item?id=37101862 https://news.ycombinator.com/item?id=37101862 Pixi is integrating uv: https://github.com/astral-sh/uv/issues/1572#issuecomment-1949979306 https://github.com/astral-sh/uv/issues/1572#issuecomment-194... astral-sh/uv [1] solves versions with PubGrub [2], which was written for Dart: [1] https://github.com/astral-sh/uv https://github.com/astral-sh/uv [2] https://github.com/pubgrub-rs/pubgrub https://github.com/pubgrub-rs/pubgrub uv's benchmarks are run with hyperfine, a CLI benchmarking tool: https://github.com/sharkdp/hyperfine https://github.com/sharkdp/hyperfine astral-sh/rye was mitsuhiko/rye, and rye wraps uv with "fallback to unearth [resolvelib] and pip-tools": https://github.com/astral-sh/rye https://github.com/astral-sh/rye https://lucumr.pocoo.org/2024/2/15/rye-grows-with-uv/ https://lucumr.pocoo.org/2024/2/15/rye-grows-with-uv/ So, TIL it looks like rye for python and pixi for conda (python, r, rust, go,) are the latest tools for python packaging, not apt packaging. fpm is one way to create deb packages from python virtualenvs for install with the new APT solver, which resulted in this research.
- matrss 3y agoI think install time dependency resolution was a bad idea and needs to be removed entirely. Packages for package managers that do it are untestable by design, because their runtime behavior cannot be known at build time. "Functional package management", as implemented in nix and guix, is a much saner approach.
- pornel 3y agoI wish Debian/APT had first-class support for having multiple versions of the same library. On a large system it's inevitable that packages will be upgrading at different rates, and a bit of duplication can be a way out of complex system-wide dependency chains held by the most outdated package. Despite lack of tooling support, Debian still does not completely avoid duplicated packages. They merely manually move the major version number to the package name (foo v10.0 and foo11 v11.0), causing package names to vary between distros and Debian versions. This seems like the worst solution, since APT can't automatically do it where it would be useful, and renamed packages complicate use of tools like pkg-config, ansible, and providing install instructions to users.
- matrss 3y agoThat is an orthogonal issue. Guix has first-class support for versions (where "version" means the version string of the source code), while nixpkgs uses the same naming-based approach as Debian. The important part is being able to install two packages in isolation, irrespective of if they represent the same software in different versions or if they are completely unrelated. Apt couldn't install two unrelated packages if they contain files that have the same path, to my knowledge. As soon as one foregos putting everything into a single filesystem structure (like in FHS, which most distros follow to some extent) the logical conclusion will be to prefix each packages directory with some identifier that meaningfully describes its content (which could be considered an "extended version" of this package). With nix this is a hash of all the inputs that went into the package (its own source code, compiler flags, build settings, dependencies, etc. and recursively the same for all of its dependencies). This means that if the compiler flags of e.g. a five levels deep transitive dependency are changed, then the package that depends on it changes as well, which is only reasonable since its behavior could change. At that point all dependency resolution has happened at build time and none is required at runtime. Being able to install different versions of the same package is merely a by-product of this approach.