3 ms·

The release of this pretty much hosed my morning at work. Suddenly CI wouldn’t build because this broke things in pyyaml and awscli. And then I got to re-lear

The release of this pretty much hosed my morning at work. Suddenly CI wouldn’t build because this broke things in pyyaml and awscli. And then I got to re-learn how very bad Poetry and python dependency management is. I’m trying to update pyyaml to 6.0.1 and Poetry systematically downloaded every single major, minor, patch version of another dependency in trying to find which would fit the version constraints. Took half an hour.

gaganyaan 3y ago
If you're using poetry, why not just add a `cython = "<3.0"` to your pyproject.toml file? You can lock down subdependency versions like that, and presumably the versions of everything that worked yesterday still work today.
- Waterluvian 3y ago
  It’s not part of my dependencies and it wasn’t on my radar. What I learned was that pyyaml had a new version that fixed the error being thrown by pyyaml.
- aeyes 3y ago
  If a top level dependency has build time dependencies then those do not respect the version locks of your pyproject.toml unless you disable build isolation.
nextlevelwizard 3y ago
Why are you blindly upgrading major versions of your tools? Sounds like fundamental design issue more than anything.
- globular-toast 3y ago
  You'd hope people would learn this time, but they'll probably just keep blaming their tools...
- gyrovagueGeist 3y ago
  Version dependencies aren’t propagated and can be outside your local control. As an example elsewhere in this thread: PyYAML did not have a pinned Cython dependency with a maximum version before this, so if your project depended on a pinned version of PyYAML it would break when trying to build it after Cython 3.0.0 released.
  - nextlevelwizard 3y ago
    Why are you constantly rebuilding dependencies? Surely you should only rebuild if there is a new (and vetted) version
    - gyrovagueGeist 3y ago
      Lmao you’d think. But this is not the default or recommended pip/setuptools behavior: https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#build-isolation https://pip.pypa.io/en/stable/reference/build-system/pyproje...
- deleted 3y ago
  [deleted]
- plonk 3y ago
  The usual methods to control versions with a lockfile don’t work here because it’s a build dependency. You’d need to disable build isolation and install the right build tools in your environment before installing pyyaml but that can become a mess real quick.
- ynik 3y ago
  It's a fundamental design issue with Python package management that makes it impossible to pin the version of build-time dependencies. You can do everything seemingly right, with pip-tools creating lockfiles containing hashes of downloaded packages, stuff seems to be stable for a year+. And then suddenly your build starts failing because the lockfiles only contain runtime-dependencies, not build-time-dependencies, and some library you are depending on just was silently downloading the latest version of some package you've never heard of, until it broke. The only semi-sane workaround I know of is to use pip's `--only-binary` option to prevent any automatic builds at package installation time. Then you usually will also need your own package server for storing precompiled wheels for those third-party-dependencies that don't published compiled packages on pypi. This way you build the packages using some random tool version only once, so if it works the first time it will keep working. In other language ecosystems were the package management isn't built on a tower of shit, this problem doesn't exist in the first place.
  - duped 3y ago
    Doesn't poetry have a solution for this?
  - nextlevelwizard 3y ago
    >you are depending on just was silently downloading the latest version of some package you've never heard of, until it broke. How? Are you running `pip install` every time you run your pipeline? Why do you need to constantly be reinstalling your packages?
raverbashing 3y ago
> Poetry systematically downloaded every single major, minor, patch version of another dependency in trying to find which would fit the version constraints. Great news! Better do that then have to do it manually But yes it sucks, still Poetry is much better than Pip (where it just gives up and tell you to go deal with the issue yourself!) Now why do people care about Cython and especially in things like dependencies I don't understand. awscli shoud be an independent package! "oh but it is faster" are you counting the time it takes to fix things when they break because you are depending on a fragile thing?
- Waterluvian 3y ago
  Other systems can do this with far less effort. It’s really a total joke.
  - raverbashing 3y ago
    Well the way npm solves this is having every package with its own copy of dependencies, maybe that's the way python should go, but I'm not sure if other package managers are much better than that (let's say Cargo in Rust)
plonk 3y ago
I think it doesn’t have a choice in downloading all versions if they use a setup.py config file. You have to get the sources to run it and get the package’s metadata.
aldanor 3y ago
And that's pretty much why you use binary package distributions like conda, despite all its flaws a sudden release of cython will never break your deps builds because there's nothing to build.
- globular-toast 3y ago
  You can force pip to only use wheels. Then if something fails, build your own wheel in your own (pinned) environment. Could be done in multi-stage docker build, for example. If one of the build deps has no wheel then you're fucked, though. At some level you have to rely on a toolchain.

gaganyaan 3y ago

If you're using poetry, why not just add a `cython = "<3.0"` to your pyproject.toml file? You can lock down subdependency versions like that, and presumably the versions of everything that worked yesterday still work today.

Waterluvian 3y ago
It’s not part of my dependencies and it wasn’t on my radar. What I learned was that pyyaml had a new version that fixed the error being thrown by pyyaml.
aeyes 3y ago
If a top level dependency has build time dependencies then those do not respect the version locks of your pyproject.toml unless you disable build isolation.

nextlevelwizard 3y ago

Why are you blindly upgrading major versions of your tools? Sounds like fundamental design issue more than anything.

globular-toast 3y ago
You'd hope people would learn this time, but they'll probably just keep blaming their tools...
gyrovagueGeist 3y ago
Version dependencies aren’t propagated and can be outside your local control. As an example elsewhere in this thread: PyYAML did not have a pinned Cython dependency with a maximum version before this, so if your project depended on a pinned version of PyYAML it would break when trying to build it after Cython 3.0.0 released.
- nextlevelwizard 3y ago
  Why are you constantly rebuilding dependencies? Surely you should only rebuild if there is a new (and vetted) version
  - gyrovagueGeist 3y ago
    Lmao you’d think. But this is not the default or recommended pip/setuptools behavior: https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#build-isolation https://pip.pypa.io/en/stable/reference/build-system/pyproje...
deleted 3y ago
[deleted]
plonk 3y ago
The usual methods to control versions with a lockfile don’t work here because it’s a build dependency. You’d need to disable build isolation and install the right build tools in your environment before installing pyyaml but that can become a mess real quick.
ynik 3y ago
It's a fundamental design issue with Python package management that makes it impossible to pin the version of build-time dependencies. You can do everything seemingly right, with pip-tools creating lockfiles containing hashes of downloaded packages, stuff seems to be stable for a year+. And then suddenly your build starts failing because the lockfiles only contain runtime-dependencies, not build-time-dependencies, and some library you are depending on just was silently downloading the latest version of some package you've never heard of, until it broke. The only semi-sane workaround I know of is to use pip's `--only-binary` option to prevent any automatic builds at package installation time. Then you usually will also need your own package server for storing precompiled wheels for those third-party-dependencies that don't published compiled packages on pypi. This way you build the packages using some random tool version only once, so if it works the first time it will keep working. In other language ecosystems were the package management isn't built on a tower of shit, this problem doesn't exist in the first place.
- duped 3y ago
  Doesn't poetry have a solution for this?
- nextlevelwizard 3y ago
  >you are depending on just was silently downloading the latest version of some package you've never heard of, until it broke. How? Are you running `pip install` every time you run your pipeline? Why do you need to constantly be reinstalling your packages?

raverbashing 3y ago

> Poetry systematically downloaded every single major, minor, patch version of another dependency in trying to find which would fit the version constraints. Great news! Better do that then have to do it manually But yes it sucks, still Poetry is much better than Pip (where it just gives up and tell you to go deal with the issue yourself!) Now why do people care about Cython and especially in things like dependencies I don't understand. awscli shoud be an independent package! "oh but it is faster" are you counting the time it takes to fix things when they break because you are depending on a fragile thing?

Waterluvian 3y ago
Other systems can do this with far less effort. It’s really a total joke.
- raverbashing 3y ago
  Well the way npm solves this is having every package with its own copy of dependencies, maybe that's the way python should go, but I'm not sure if other package managers are much better than that (let's say Cargo in Rust)

plonk 3y ago

I think it doesn’t have a choice in downloading all versions if they use a setup.py config file. You have to get the sources to run it and get the package’s metadata.

aldanor 3y ago

And that's pretty much why you use binary package distributions like conda, despite all its flaws a sudden release of cython will never break your deps builds because there's nothing to build.

globular-toast 3y ago
You can force pip to only use wheels. Then if something fails, build your own wheel in your own (pinned) environment. Could be done in multi-stage docker build, for example. If one of the build deps has no wheel then you're fucked, though. At some level you have to rely on a toolchain.