6 ms·

I use arch on all my machines and there are really a lot of ways it has broken. - Arch will release a new version of Xorg before the graphics card vendors rele

I use arch on all my machines and there are really a lot of ways it has broken. - Arch will release a new version of Xorg before the graphics card vendors release new drivers that are compatible with it. Same for kernel versions too. - I've had gedit start crashing in new minor versions of gnome due to a setting being incompatible, needing to track that down and unset - If you have python virtualenvs for development and system python is upgraded to a new major version, all your virtualenvs break - If arch upgrades the major version of glibc during some random package install (rather than a system upgrade), and you don't upgrade the whole system at once, every app that didn't get upgraded will fail to start due to soname mismatches... and that can mean that pacman, sudo, etc are all busted (this has actually happened to me) - If you do a full system upgrade and have AUR stuff installed, you need to be sure to upgrade the AUR stuff otherwise it could break due to being incompatible with any library that was upgraded. - In general (not specific to arch linux), new versions of software break stuff all the time. You tend to hit way more of this on arch if you keep your system up to date.

formerly_proven 5y ago
> - If arch upgrades the major version of glibc during some random package install (rather than a system upgrade), and you don't upgrade the whole system at once, every app that didn't get upgraded will fail to start due to soname mismatches... and that can mean that pacman, sudo, etc are all busted (this has actually happened to me) Okay I just wanna point something out here, partial upgrades in Arch are broken by design because they simply can't work. Don't do it. That being said... > - Arch will release a new version of Xorg before the graphics card vendors release new drivers that are compatible with it. Same for kernel versions too. ... kernel ABI is stable and even ancillary interfaces are usually stable, so usually it's quite A-OK to not immediately upgrade the kernel.
- Liskni_si 5y ago
  Can you elaborate why has Arch chosen to not support partial upgrades by design? They do work in Debian and it's saved my day a number of times. They let me roll back a broken upgrade without holding back upgrades of the rest of the system, and my computer then continues to be usable until the package in question is fixed.
  - Quekid5 5y ago
    They choose not to officially support it precisely because of the types of issues that upgrading e.g. glibc or openssl can cause. That said, the vast majority of the time only upgrading a particular application/package (and its dependencies) will work just fine. It's just that there are no (official) guarantees.
    - Liskni_si 5y ago
      The types of issues upgrading e.g. glibc or openssl can cause are largely predictable, aren't they? That's why there's libtool and sonames and that kind of stuff. The only missing piece is support for this in the package manager…
      - nemetroid 5y ago
        Pacman can roll back packages, and does let you stop certain packages from being upgraded. As you say, you could probably get away with doing it, if the package in question has "boring" (i.e. ABI stable) dependencies.
  - wvh 5y ago
    Because Debian sort of maintains their own "pinned" versions between upstream and you, you basically get Debian's version possibly with their own patches and fixes; whereas Arch just directly gives you upstream, compiled for your platform. Debian's way of working is labour intensive, requiring packagers to fork, follow and maintain fixes and security issues in the version of the software they're packaging. They generally do a great job, but this is not a sustainable approach for smaller distributions. Arch Linux on the other hand follows upstream in real time, so you get the latest fixes directly from upstream, but there is no real Arch Linux "version buffer" that allows to freeze the versions of (parts of) your system. You move with the stream, that's sort of the philosophy of rolling distributions like Arch Linux.
    - Liskni_si 5y ago
      I don't think that's it, really. The real trick is that Debian packages have sonames in their versions, so ABI compatibility is encoded in package dependencies. So when I try to downgrade a library (to a known older version from snapshot.debian.org), it knows precisely which packages depend on the new version and forces them to be downgraded as well. This is entirely orthogonal to following upstream vs. backporting patches. Sure, Debian (stable) does backport patches, which makes it more likely that single package downgrades don't downgrade half of the system, but it really is a different thing. Debian testing/unstable follow upstream to a larger extent than Debian stable, and upstream fixes are usually preferred to patch backports. Still, partial upgrades and downgrades almost always work without trouble.
- paulfurtado 5y ago
  Yeah, definitely usually fine to avoid kernel upgrades, so that is a saving grace. The issue is just that pacman -Syu is going to be full of surprises
nemetroid 5y ago
Thanks. > - Arch will release a new version of Xorg before the graphics card vendors release new drivers that are compatible with it. Same for kernel versions too. I imagine this is mainly a problem if you have an Nvidia card (which would explain why I haven't had the problem). > - In general (not specific to arch linux), new versions of software break stuff all the time. You tend to hit way more of this on arch if you keep your system up to date. More often, sure. But for software delivery from the production side, the common viewpoint seems to be that deploying more often leads to less pain in total, because you're making smaller increments so the individual failures are smaller as well. As such, aside from integration issues like Xorg and the drivers, I would expect Arch to have fewer major breakages than (e.g.) using LTS releases of some distro and upgrading every second year.
- paulfurtado 5y ago
  > More often, sure. But for software delivery from the production side, the common viewpoint seems to be that deploying more often leads to less pain in total, because you're making smaller increments so the individual failures are smaller as well. > > As such, aside from integration issues like Xorg and the drivers, I would expect Arch to have fewer major breakages than (e.g.) using LTS releases of some distro and upgrading every second year. This is mostly true for production infra (at least for your own software in production infra) but I'm not sure this logic flies for personal machines and software you don't directly interact with. I have used arch for at least 7 years and I have had upgrades render my system unbootable (or nearly unbootable, like Xorg/GDM/Gnome failing to start). These issues were mostly not due to my configuration needing to change, they were due to breakages between different packages that would end up being fixed in some later minor version. As an end-user upgrading distros every 2 years, I really doubt I'd hit many major breakages each time as all of those pieces of software will have been in the wild for a bit and major bugs will have likely been fixed. I think the system-level issues that are resolved by frequent deploys are stuff like "systemd deprecated setting X in service unit files, so I need to update my config" or "library xyz changed some API so I need to update my app", etc. With linux distros that have coordinated releases like fedora/ubuntu/debian/etc my software's interaction with the distro may break, but for the most part the major inter-package relationships within the distro get some amount of coordinated testing which doesn't happen with rolling release distros. Put another way: deploying more often is great for quickly discovering problems introduced in software that you write. Deploying every single dependency in a linux distro more often is going to cause you to hit every bug destined to be fixed in some minor version of the software in pieces of code that you are very far away from and lack the context to quickly debug. So you will hit a much higher sum total of bugs that would have otherwise ended up getting fixed whether you personally hit them or not. However, if you are developing a linux distribution itself, then yes your CI infra should be constantly upgrading dependencies. I'm not convinced that an arch-paced rolling release at a distro level will ever reach a point where inter-package dependencies do not cause totally unexpected bugs given just how many inter-library dependencies there are. The entire OSS ecosystem would need to write a ton more tests for this to be a reality, and distros would need to run that full suite of tests any time and dependency is upgraded. And even then, there's still so much possibility for breakages given that shared libraries don't do a fantastic job of versioning and the upstream vendors can't possibly ensure their software works against versions of a library they never ran it against. There's a linus torvalds post about this: https://lwn.net/ml/linux-kernel/CAHk-=whs8QZf3YnifdLv57+FhBi5_WeNTG1B-suOES=RcUSmQg@mail.gmail.com/ https://lwn.net/ml/linux-kernel/CAHk-=whs8QZf3YnifdLv57+FhBi... Despite all of this, Arch is actually amazingly stable, and the entire community is probably better off due to the existence of arch and arch users. We might be the best integration test there is for OSS software.
mixedCase 5y ago
I'm sure you know all this, but just to give other people some context on how Arch expects one as a user to handle this: > - Arch will release a new version of Xorg before the graphics card vendors release new drivers that are compatible with it. Same for kernel versions too. You accidentally put a plural in "vendors", but in practice this is just Nvidia. If you are forced by circumstance to deal with an Nvidia card, use the LTS kernel and most of your problems go away, and you can also pin your X.org version and manually update it. The real solution is to use a GPU vendor that has mainline drivers, the only valid reason nowadays not to do that being CUDA or being unable to acquire hardware with mainline support. > - I've had gedit start crashing in new minor versions of gnome due to a setting being incompatible, needing to track that down and unset That's a gedit bug. Complain to gedit devs or stop using it. > - If you have python virtualenvs for development and system python is upgraded to a new major version, all your virtualenvs break You should not be using system python for non-system tasks. Use asdf, Nix, Docker, pyenv or a similar tool for projects requiring their own non-system environment. > - If arch upgrades the major version of[...] Partial upgrades are explicitly unsupported by the distro. Pacman allows you to do this, but you're on your own. > - If you do a full system upgrade and have AUR stuff installed AUR is not Arch, it's up to each AUR maintainer to keep their scripts up to date and you as a user to keep up with those external dependencies. A common approach is to use an AUR helper to handle both system and AUR upgrades. > - In general (not specific to arch linux), new versions of software break stuff all the time. You tend to hit way more of this on arch if you keep your system up to date. And this is why as an Arch user you will be nudged by experience to use software that gives a crap about quality.
- paulfurtado 5y ago
  Yes, all of this is definitely true and I'm not actually complaining about it, I have used arch on all my machines for 7 or 8 years and don't plan to change that anytime soon, but these problems are actually unique to using arch and other rolling distros. Ubuntu, Fedora, etc have coordinated releases with some level of testing that things are compatible with each other. On arch, these are problems one must deal with though. I don't mind dealing with it, I live for Linux stuff, but the point is that it does happen to real users and it's not for everyone. > Partial upgrades are explicitly unsupported by the distro. Pacman allows you to do this, but you're on your own. Definitely correct, the issue is when you're trying to install a package to get something done, and suddenly your faced with the proposition of either upgrading your whole system while trying to get work done or attempting to upgrade just the required libraries. The latter often carries less risk, but occasionally is very problematic. > And this is why as an Arch user you will be nudged by experience to use software that gives a crap about quality. The issue is that this is true of so many very major pieces of linux software. Upstream vendors don't necessarily test their stuff in a ton of contexts, plenty of bugs occur in the kernel/xorg/wayland/gnome/glibc/python/etc. All of these are very mainstream projects that are difficult to avoid.
DyslexicAtheist 5y ago
> - If arch upgrades the major version of glibc during some random package install (rather than a system upgrade), and you don't upgrade the whole system at once this was what convinced me to switch to Debian back in the 90ies. It was the only distribution that could do this without throwing a tantrum. I was coming from RedHat and SuSE and both had huge problems with this. Debian presented a higher learning curve (at least that was what people said). Debian really only lagged behind back then with their graphical installer and the overall lack of integrated DE back then (compared to others). I really like the Arch wiki as a Debian user but never used Arch itself. Not going to change because old dog, new tricks ...

formerly_proven 5y ago

> - If arch upgrades the major version of glibc during some random package install (rather than a system upgrade), and you don't upgrade the whole system at once, every app that didn't get upgraded will fail to start due to soname mismatches... and that can mean that pacman, sudo, etc are all busted (this has actually happened to me) Okay I just wanna point something out here, partial upgrades in Arch are broken by design because they simply can't work. Don't do it. That being said... > - Arch will release a new version of Xorg before the graphics card vendors release new drivers that are compatible with it. Same for kernel versions too. ... kernel ABI is stable and even ancillary interfaces are usually stable, so usually it's quite A-OK to not immediately upgrade the kernel.

Liskni_si 5y ago
Can you elaborate why has Arch chosen to not support partial upgrades by design? They do work in Debian and it's saved my day a number of times. They let me roll back a broken upgrade without holding back upgrades of the rest of the system, and my computer then continues to be usable until the package in question is fixed.
- Quekid5 5y ago
  They choose not to officially support it precisely because of the types of issues that upgrading e.g. glibc or openssl can cause. That said, the vast majority of the time only upgrading a particular application/package (and its dependencies) will work just fine. It's just that there are no (official) guarantees.
  - Liskni_si 5y ago
    The types of issues upgrading e.g. glibc or openssl can cause are largely predictable, aren't they? That's why there's libtool and sonames and that kind of stuff. The only missing piece is support for this in the package manager…
    - nemetroid 5y ago
      Pacman can roll back packages, and does let you stop certain packages from being upgraded. As you say, you could probably get away with doing it, if the package in question has "boring" (i.e. ABI stable) dependencies.
- wvh 5y ago
  Because Debian sort of maintains their own "pinned" versions between upstream and you, you basically get Debian's version possibly with their own patches and fixes; whereas Arch just directly gives you upstream, compiled for your platform. Debian's way of working is labour intensive, requiring packagers to fork, follow and maintain fixes and security issues in the version of the software they're packaging. They generally do a great job, but this is not a sustainable approach for smaller distributions. Arch Linux on the other hand follows upstream in real time, so you get the latest fixes directly from upstream, but there is no real Arch Linux "version buffer" that allows to freeze the versions of (parts of) your system. You move with the stream, that's sort of the philosophy of rolling distributions like Arch Linux.
  - Liskni_si 5y ago
    I don't think that's it, really. The real trick is that Debian packages have sonames in their versions, so ABI compatibility is encoded in package dependencies. So when I try to downgrade a library (to a known older version from snapshot.debian.org), it knows precisely which packages depend on the new version and forces them to be downgraded as well. This is entirely orthogonal to following upstream vs. backporting patches. Sure, Debian (stable) does backport patches, which makes it more likely that single package downgrades don't downgrade half of the system, but it really is a different thing. Debian testing/unstable follow upstream to a larger extent than Debian stable, and upstream fixes are usually preferred to patch backports. Still, partial upgrades and downgrades almost always work without trouble.
paulfurtado 5y ago
Yeah, definitely usually fine to avoid kernel upgrades, so that is a saving grace. The issue is just that pacman -Syu is going to be full of surprises

nemetroid 5y ago

Thanks. > - Arch will release a new version of Xorg before the graphics card vendors release new drivers that are compatible with it. Same for kernel versions too. I imagine this is mainly a problem if you have an Nvidia card (which would explain why I haven't had the problem). > - In general (not specific to arch linux), new versions of software break stuff all the time. You tend to hit way more of this on arch if you keep your system up to date. More often, sure. But for software delivery from the production side, the common viewpoint seems to be that deploying more often leads to less pain in total, because you're making smaller increments so the individual failures are smaller as well. As such, aside from integration issues like Xorg and the drivers, I would expect Arch to have fewer major breakages than (e.g.) using LTS releases of some distro and upgrading every second year.

paulfurtado 5y ago
> More often, sure. But for software delivery from the production side, the common viewpoint seems to be that deploying more often leads to less pain in total, because you're making smaller increments so the individual failures are smaller as well. > > As such, aside from integration issues like Xorg and the drivers, I would expect Arch to have fewer major breakages than (e.g.) using LTS releases of some distro and upgrading every second year. This is mostly true for production infra (at least for your own software in production infra) but I'm not sure this logic flies for personal machines and software you don't directly interact with. I have used arch for at least 7 years and I have had upgrades render my system unbootable (or nearly unbootable, like Xorg/GDM/Gnome failing to start). These issues were mostly not due to my configuration needing to change, they were due to breakages between different packages that would end up being fixed in some later minor version. As an end-user upgrading distros every 2 years, I really doubt I'd hit many major breakages each time as all of those pieces of software will have been in the wild for a bit and major bugs will have likely been fixed. I think the system-level issues that are resolved by frequent deploys are stuff like "systemd deprecated setting X in service unit files, so I need to update my config" or "library xyz changed some API so I need to update my app", etc. With linux distros that have coordinated releases like fedora/ubuntu/debian/etc my software's interaction with the distro may break, but for the most part the major inter-package relationships within the distro get some amount of coordinated testing which doesn't happen with rolling release distros. Put another way: deploying more often is great for quickly discovering problems introduced in software that you write. Deploying every single dependency in a linux distro more often is going to cause you to hit every bug destined to be fixed in some minor version of the software in pieces of code that you are very far away from and lack the context to quickly debug. So you will hit a much higher sum total of bugs that would have otherwise ended up getting fixed whether you personally hit them or not. However, if you are developing a linux distribution itself, then yes your CI infra should be constantly upgrading dependencies. I'm not convinced that an arch-paced rolling release at a distro level will ever reach a point where inter-package dependencies do not cause totally unexpected bugs given just how many inter-library dependencies there are. The entire OSS ecosystem would need to write a ton more tests for this to be a reality, and distros would need to run that full suite of tests any time and dependency is upgraded. And even then, there's still so much possibility for breakages given that shared libraries don't do a fantastic job of versioning and the upstream vendors can't possibly ensure their software works against versions of a library they never ran it against. There's a linus torvalds post about this: https://lwn.net/ml/linux-kernel/CAHk-=whs8QZf3YnifdLv57+FhBi5_WeNTG1B-suOES=RcUSmQg@mail.gmail.com/ https://lwn.net/ml/linux-kernel/CAHk-=whs8QZf3YnifdLv57+FhBi... Despite all of this, Arch is actually amazingly stable, and the entire community is probably better off due to the existence of arch and arch users. We might be the best integration test there is for OSS software.

mixedCase 5y ago

I'm sure you know all this, but just to give other people some context on how Arch expects one as a user to handle this: > - Arch will release a new version of Xorg before the graphics card vendors release new drivers that are compatible with it. Same for kernel versions too. You accidentally put a plural in "vendors", but in practice this is just Nvidia. If you are forced by circumstance to deal with an Nvidia card, use the LTS kernel and most of your problems go away, and you can also pin your X.org version and manually update it. The real solution is to use a GPU vendor that has mainline drivers, the only valid reason nowadays not to do that being CUDA or being unable to acquire hardware with mainline support. > - I've had gedit start crashing in new minor versions of gnome due to a setting being incompatible, needing to track that down and unset That's a gedit bug. Complain to gedit devs or stop using it. > - If you have python virtualenvs for development and system python is upgraded to a new major version, all your virtualenvs break You should not be using system python for non-system tasks. Use asdf, Nix, Docker, pyenv or a similar tool for projects requiring their own non-system environment. > - If arch upgrades the major version of[...] Partial upgrades are explicitly unsupported by the distro. Pacman allows you to do this, but you're on your own. > - If you do a full system upgrade and have AUR stuff installed AUR is not Arch, it's up to each AUR maintainer to keep their scripts up to date and you as a user to keep up with those external dependencies. A common approach is to use an AUR helper to handle both system and AUR upgrades. > - In general (not specific to arch linux), new versions of software break stuff all the time. You tend to hit way more of this on arch if you keep your system up to date. And this is why as an Arch user you will be nudged by experience to use software that gives a crap about quality.

paulfurtado 5y ago
Yes, all of this is definitely true and I'm not actually complaining about it, I have used arch on all my machines for 7 or 8 years and don't plan to change that anytime soon, but these problems are actually unique to using arch and other rolling distros. Ubuntu, Fedora, etc have coordinated releases with some level of testing that things are compatible with each other. On arch, these are problems one must deal with though. I don't mind dealing with it, I live for Linux stuff, but the point is that it does happen to real users and it's not for everyone. > Partial upgrades are explicitly unsupported by the distro. Pacman allows you to do this, but you're on your own. Definitely correct, the issue is when you're trying to install a package to get something done, and suddenly your faced with the proposition of either upgrading your whole system while trying to get work done or attempting to upgrade just the required libraries. The latter often carries less risk, but occasionally is very problematic. > And this is why as an Arch user you will be nudged by experience to use software that gives a crap about quality. The issue is that this is true of so many very major pieces of linux software. Upstream vendors don't necessarily test their stuff in a ton of contexts, plenty of bugs occur in the kernel/xorg/wayland/gnome/glibc/python/etc. All of these are very mainstream projects that are difficult to avoid.

DyslexicAtheist 5y ago

> - If arch upgrades the major version of glibc during some random package install (rather than a system upgrade), and you don't upgrade the whole system at once this was what convinced me to switch to Debian back in the 90ies. It was the only distribution that could do this without throwing a tantrum. I was coming from RedHat and SuSE and both had huge problems with this. Debian presented a higher learning curve (at least that was what people said). Debian really only lagged behind back then with their graphical installer and the overall lack of integrated DE back then (compared to others). I really like the Arch wiki as a Debian user but never used Arch itself. Not going to change because old dog, new tricks ...