6 ms·

You described third party repositories and mismatching Python versions. That is a hacked up system, by definition.

by rlpb 5y ago

generalizations 5y ago
Seems uncommon that those things would prevent upgrades. It sounds more like Ubuntu is beginning to suffer from an excess of complexity.
- rlpb 5y ago
  I think part of the problem is that third party apt repositories have become normalised as if they're a reasonable way to "add on" to your existing system. In reality they're a hack on a packaging system that was never designed for third party plugability in this way. They very often break future upgrades. Third party apt repositories fundamentally cannot express all the necessary metadata to allow for upgrades to work in the general case. They don't even namespace properly; apt cannot even tell the difference between a third party installed package and one that came from the distribution. And third party packagers typically don't even consider future upgrades. This is why Ubuntu is working on snaps: they are a mechanism that allows for third party plugability in a way that does not break the system. If we want upgrades not to break, the system dpkg/apt installation needs to be left alone and not be interfered with by third parties.
  - the__alchemist 5y ago
    This is at the core of why I've found Ubuntu (More broadly Linux?) to be frustrating to use. Installing programs or functionality beyond what it ships with leads to the system ending in a "hacked up", as you put it, state. Then reliability tanks.
  - KptMarchewa 5y ago
    So, what should I do if you don't decide to include critical release of a package that was out there for 6 months? https://github.com/swaywm/sway/releases https://github.com/swaywm/sway/releases
    - rlpb 5y ago
      Here's some further information on sway specifically. Looks like it couldn't be updated because depending packages weren't ready for the switch: https://bugs.launchpad.net/ubuntu/+source/phoc/+bug/1938404 https://bugs.launchpad.net/ubuntu/+source/phoc/+bug/1938404
    - rlpb 5y ago
      Sorry that wasn't included. Please understand that it's not about "you" and "me" though. Anyone can contribute to Ubuntu, and "any responsibility can be carried by any contributor who demonstrates the required capacity and competence" (https://ubuntu.com/community/code-of-conduct https://ubuntu.com/community/code-of-conduct). So please join us and help to make this happen next time. Of course there are processes and policies to help us ensure quality and that other user expectations (eg. stability, integration) are met as best as we can. These take time and effort to negotiate. But we'd be happy to help to explain what's needed to be contributed to make progress in Ubuntu itself for any specific circumstance. Find us in #ubuntu-devel on Libera Chat, or https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discu...
  - pxc 5y ago
    > Third party apt repositories [...] don't even namespace properly; apt cannot even tell the difference between a third party installed package and one that came from the distribution. This is true. dnf and zypper handle this just fine in the RPM world, and they operate under basically the same paradigm. Are there no plans for apt to catch up?
deleted 5y ago
[deleted]
josephg 5y ago
I don’t know much about python, but for nodejs dev work I have about half a dozen versions of nodejs installed. It’s super handy having all those versions installed so I can test that a certain native nodejs package I develop builds and works properly. And to rule out nodejs itself when reproducing bugs. I use nvm (or fvm) or something to manage all that because the system package managers updated too slowly. What are you saying here? That having multiple versions of python installed can put your whole OS into some sort of unrecoverable corrupted state, and that’s expected and ok? That’s a non starter for me.
- md8z 5y ago
  If you replace the version of python that the upgrade process expects, then yeah, you'll probably run into problems. The upgrade process can't possibly support every single version of any given package, that would defeat the entire purpose of having a stable release cycle. If it's a core package like python which sits really close to the top of the dependency tree then the effects of swapping that out will cascade downward and that's why everything breaks so badly. In my experience, this is the only safe way to have third party apt repositories: - Don't install any third party packages which have reverse dependencies. This means no libraries, only applications. If you need newer libraries as a dependency then you have to statically compile the application. - Don't install any third party packages which have the same name as an upstream package. If you want to install a newer third party version of an upstream package then you need to change the name of it and have it install all of its files in different locations. A prefix change is enough here, /opt is a popular choice. - Delete all third party packages before doing an upgrade, in case a new upstream package happens to have the same name, or needs to put files in the same locations. Do anything outside of that and you will risk getting your install into a broken state. Because even though you might think you are just installing some applications, what you are really doing is bolting things on to the upgrade process for the entire core system. Using nvm is perfectly safe though because that just installs the different versions to your home directory, it doesn't touch apt.
  - josephg 5y ago
    That makes sense; though to be clear, you're essentially saying that apt isn't appropriate for 3rd party packages. And that: - The question of which files go in which directories in the system is a question for canonical (or whoever) to decide. Not end users or 3rd party developers - The namespace of apt package names is special, and 3rd party / external dpkgs shouldn't add anything into it I think this stance is entirely reasonable. Its essentially what windows and macos both do (and to increasing degrees with each version). But its a pity its taken the current apt packaging disaster to realise it. Its really convenient for users to have a package manager, and for 3rd party developers be able to write software for it. And these problems seem solvable: - Namespace all 3rd party packages. Eg, @apple/foundationdb-server (as distinct from @canonical/git) - Use overlayfs or something to essentially make /usr, /usr/bin, etc read only by normal processes. If these paths were only writable by system packages, the whole system would be much more predictable (and verifyable) - since that filesystem would only - could only contain a subset of the files in apt. I get that flatpak and snap are both attempts to replace apt. But its a pity they're needed. I'm enjoying mint, but if I ever rebuild this machine I'm going to give nix a try.
capableweb 5y ago
Yes, because Ubuntu doesn't ship everything you need, third-party repositories are needed for my day-to-day job. None of the packages I get from that are overwriting existing software from official repositories, but still Ubuntu recommends turning those off, not sure how that should break an upgrade. On the other hand, my Arch installation runs with tons of custom software, with repositories and more added to it, but still somehow can handle upgrades 100 times better than Ubuntu. Regarding the python, the error message was something like "python is gonna be python 3 after the upgrade, and we can't handle that so you should uninstall python before upgrading", or something similar. I'm just using what Canonical/Ubuntu are providing me when it comes to Python, but still somehow it's up to me to remove/install stuff manually when it comes to upgrading.
- jeroenhd 5y ago
  I use Manjaro and different Python versions alongside each other regularly break my updates because all of them want to be the system python3 binary at the same time. I've had to nuke and reinstall tons of software because of version clashes, sometimes needing me to cast arcane package manager spells which every manual tells you to never ever do. Hell, a lot of really common AUR packages or dependencies sometimes just plain fail to compile. That's the danger of the AUR, I guess, but there's no real difference between the AUR and PPAs/Ubuntu repositories. At least Manjaro pins packages for a short while before releasing them onto my laptop, I can't imagine the stress an average Arch update would cause me. Ubuntu is not a system designed for running multiple versions of packages like you want it to. You can make it do it if you really want to, but you can't expect the Ubuntu developers to provide support for your unique use case. They give you one version of Python, the version that the _entire system_ expects, and if you're not happy with that, you should probably move away from Ubuntu. It can't and probably won't satisfy your day-to-day job's requirements. Try a rolling distro instead, or run the Nix package manager next to the system package manager so you can work around the lack of support Ubuntu provides you. In my opinion, it's a lot better to abort the upgrade than to knowingly break your system and make you suffer through the recovery process. Sure, the installer could ignore the problem, or remove your carefully crafted Python setup, but you'd still be complaining if it did that and you had to reconfigure your entire machine again.

You described third party repositories and mismatching Python versions. That is a hacked up system, by definition.

by rlpb 5y ago

You described third party repositories and mismatching Python versions. That is a hacked up system, by definition.

Seems uncommon that those things would prevent upgrades. It sounds more like Ubuntu is beginning to suffer from an excess of complexity.

rlpb 5y ago
I think part of the problem is that third party apt repositories have become normalised as if they're a reasonable way to "add on" to your existing system. In reality they're a hack on a packaging system that was never designed for third party plugability in this way. They very often break future upgrades. Third party apt repositories fundamentally cannot express all the necessary metadata to allow for upgrades to work in the general case. They don't even namespace properly; apt cannot even tell the difference between a third party installed package and one that came from the distribution. And third party packagers typically don't even consider future upgrades. This is why Ubuntu is working on snaps: they are a mechanism that allows for third party plugability in a way that does not break the system. If we want upgrades not to break, the system dpkg/apt installation needs to be left alone and not be interfered with by third parties.
- the__alchemist 5y ago
  This is at the core of why I've found Ubuntu (More broadly Linux?) to be frustrating to use. Installing programs or functionality beyond what it ships with leads to the system ending in a "hacked up", as you put it, state. Then reliability tanks.
- KptMarchewa 5y ago
  So, what should I do if you don't decide to include critical release of a package that was out there for 6 months? https://github.com/swaywm/sway/releases https://github.com/swaywm/sway/releases
  - rlpb 5y ago
    Here's some further information on sway specifically. Looks like it couldn't be updated because depending packages weren't ready for the switch: https://bugs.launchpad.net/ubuntu/+source/phoc/+bug/1938404 https://bugs.launchpad.net/ubuntu/+source/phoc/+bug/1938404
  - rlpb 5y ago
    Sorry that wasn't included. Please understand that it's not about "you" and "me" though. Anyone can contribute to Ubuntu, and "any responsibility can be carried by any contributor who demonstrates the required capacity and competence" (https://ubuntu.com/community/code-of-conduct https://ubuntu.com/community/code-of-conduct). So please join us and help to make this happen next time. Of course there are processes and policies to help us ensure quality and that other user expectations (eg. stability, integration) are met as best as we can. These take time and effort to negotiate. But we'd be happy to help to explain what's needed to be contributed to make progress in Ubuntu itself for any specific circumstance. Find us in #ubuntu-devel on Libera Chat, or https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discu...
- pxc 5y ago
  > Third party apt repositories [...] don't even namespace properly; apt cannot even tell the difference between a third party installed package and one that came from the distribution. This is true. dnf and zypper handle this just fine in the RPM world, and they operate under basically the same paradigm. Are there no plans for apt to catch up?

deleted 5y ago

[deleted]

josephg 5y ago

I don’t know much about python, but for nodejs dev work I have about half a dozen versions of nodejs installed. It’s super handy having all those versions installed so I can test that a certain native nodejs package I develop builds and works properly. And to rule out nodejs itself when reproducing bugs. I use nvm (or fvm) or something to manage all that because the system package managers updated too slowly. What are you saying here? That having multiple versions of python installed can put your whole OS into some sort of unrecoverable corrupted state, and that’s expected and ok? That’s a non starter for me.

md8z 5y ago
If you replace the version of python that the upgrade process expects, then yeah, you'll probably run into problems. The upgrade process can't possibly support every single version of any given package, that would defeat the entire purpose of having a stable release cycle. If it's a core package like python which sits really close to the top of the dependency tree then the effects of swapping that out will cascade downward and that's why everything breaks so badly. In my experience, this is the only safe way to have third party apt repositories: - Don't install any third party packages which have reverse dependencies. This means no libraries, only applications. If you need newer libraries as a dependency then you have to statically compile the application. - Don't install any third party packages which have the same name as an upstream package. If you want to install a newer third party version of an upstream package then you need to change the name of it and have it install all of its files in different locations. A prefix change is enough here, /opt is a popular choice. - Delete all third party packages before doing an upgrade, in case a new upstream package happens to have the same name, or needs to put files in the same locations. Do anything outside of that and you will risk getting your install into a broken state. Because even though you might think you are just installing some applications, what you are really doing is bolting things on to the upgrade process for the entire core system. Using nvm is perfectly safe though because that just installs the different versions to your home directory, it doesn't touch apt.
- josephg 5y ago
  That makes sense; though to be clear, you're essentially saying that apt isn't appropriate for 3rd party packages. And that: - The question of which files go in which directories in the system is a question for canonical (or whoever) to decide. Not end users or 3rd party developers - The namespace of apt package names is special, and 3rd party / external dpkgs shouldn't add anything into it I think this stance is entirely reasonable. Its essentially what windows and macos both do (and to increasing degrees with each version). But its a pity its taken the current apt packaging disaster to realise it. Its really convenient for users to have a package manager, and for 3rd party developers be able to write software for it. And these problems seem solvable: - Namespace all 3rd party packages. Eg, @apple/foundationdb-server (as distinct from @canonical/git) - Use overlayfs or something to essentially make /usr, /usr/bin, etc read only by normal processes. If these paths were only writable by system packages, the whole system would be much more predictable (and verifyable) - since that filesystem would only - could only contain a subset of the files in apt. I get that flatpak and snap are both attempts to replace apt. But its a pity they're needed. I'm enjoying mint, but if I ever rebuild this machine I'm going to give nix a try.

capableweb 5y ago

Yes, because Ubuntu doesn't ship everything you need, third-party repositories are needed for my day-to-day job. None of the packages I get from that are overwriting existing software from official repositories, but still Ubuntu recommends turning those off, not sure how that should break an upgrade. On the other hand, my Arch installation runs with tons of custom software, with repositories and more added to it, but still somehow can handle upgrades 100 times better than Ubuntu. Regarding the python, the error message was something like "python is gonna be python 3 after the upgrade, and we can't handle that so you should uninstall python before upgrading", or something similar. I'm just using what Canonical/Ubuntu are providing me when it comes to Python, but still somehow it's up to me to remove/install stuff manually when it comes to upgrading.

jeroenhd 5y ago
I use Manjaro and different Python versions alongside each other regularly break my updates because all of them want to be the system python3 binary at the same time. I've had to nuke and reinstall tons of software because of version clashes, sometimes needing me to cast arcane package manager spells which every manual tells you to never ever do. Hell, a lot of really common AUR packages or dependencies sometimes just plain fail to compile. That's the danger of the AUR, I guess, but there's no real difference between the AUR and PPAs/Ubuntu repositories. At least Manjaro pins packages for a short while before releasing them onto my laptop, I can't imagine the stress an average Arch update would cause me. Ubuntu is not a system designed for running multiple versions of packages like you want it to. You can make it do it if you really want to, but you can't expect the Ubuntu developers to provide support for your unique use case. They give you one version of Python, the version that the _entire system_ expects, and if you're not happy with that, you should probably move away from Ubuntu. It can't and probably won't satisfy your day-to-day job's requirements. Try a rolling distro instead, or run the Nix package manager next to the system package manager so you can work around the lack of support Ubuntu provides you. In my opinion, it's a lot better to abort the upgrade than to knowingly break your system and make you suffer through the recovery process. Sure, the installer could ignore the problem, or remove your carefully crafted Python setup, but you'd still be complaining if it did that and you had to reconfigure your entire machine again.