4 ms·

Why are you blindly upgrading major versions of your tools? Sounds like fundamental design issue more than anything.

globular-toast 3y ago
You'd hope people would learn this time, but they'll probably just keep blaming their tools...
gyrovagueGeist 3y ago
Version dependencies aren’t propagated and can be outside your local control. As an example elsewhere in this thread: PyYAML did not have a pinned Cython dependency with a maximum version before this, so if your project depended on a pinned version of PyYAML it would break when trying to build it after Cython 3.0.0 released.
- nextlevelwizard 3y ago
  Why are you constantly rebuilding dependencies? Surely you should only rebuild if there is a new (and vetted) version
  - gyrovagueGeist 3y ago
    Lmao you’d think. But this is not the default or recommended pip/setuptools behavior: https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#build-isolation https://pip.pypa.io/en/stable/reference/build-system/pyproje...
deleted 3y ago
[deleted]
plonk 3y ago
The usual methods to control versions with a lockfile don’t work here because it’s a build dependency. You’d need to disable build isolation and install the right build tools in your environment before installing pyyaml but that can become a mess real quick.
ynik 3y ago
It's a fundamental design issue with Python package management that makes it impossible to pin the version of build-time dependencies. You can do everything seemingly right, with pip-tools creating lockfiles containing hashes of downloaded packages, stuff seems to be stable for a year+. And then suddenly your build starts failing because the lockfiles only contain runtime-dependencies, not build-time-dependencies, and some library you are depending on just was silently downloading the latest version of some package you've never heard of, until it broke. The only semi-sane workaround I know of is to use pip's `--only-binary` option to prevent any automatic builds at package installation time. Then you usually will also need your own package server for storing precompiled wheels for those third-party-dependencies that don't published compiled packages on pypi. This way you build the packages using some random tool version only once, so if it works the first time it will keep working. In other language ecosystems were the package management isn't built on a tower of shit, this problem doesn't exist in the first place.
- duped 3y ago
  Doesn't poetry have a solution for this?
- nextlevelwizard 3y ago
  >you are depending on just was silently downloading the latest version of some package you've never heard of, until it broke. How? Are you running `pip install` every time you run your pipeline? Why do you need to constantly be reinstalling your packages?

You'd hope people would learn this time, but they'll probably just keep blaming their tools...

Version dependencies aren’t propagated and can be outside your local control. As an example elsewhere in this thread: PyYAML did not have a pinned Cython dependency with a maximum version before this, so if your project depended on a pinned version of PyYAML it would break when trying to build it after Cython 3.0.0 released.

nextlevelwizard 3y ago
Why are you constantly rebuilding dependencies? Surely you should only rebuild if there is a new (and vetted) version
- gyrovagueGeist 3y ago
  Lmao you’d think. But this is not the default or recommended pip/setuptools behavior: https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#build-isolation https://pip.pypa.io/en/stable/reference/build-system/pyproje...

deleted 3y ago

[deleted]

plonk 3y ago

The usual methods to control versions with a lockfile don’t work here because it’s a build dependency. You’d need to disable build isolation and install the right build tools in your environment before installing pyyaml but that can become a mess real quick.

ynik 3y ago

It's a fundamental design issue with Python package management that makes it impossible to pin the version of build-time dependencies. You can do everything seemingly right, with pip-tools creating lockfiles containing hashes of downloaded packages, stuff seems to be stable for a year+. And then suddenly your build starts failing because the lockfiles only contain runtime-dependencies, not build-time-dependencies, and some library you are depending on just was silently downloading the latest version of some package you've never heard of, until it broke. The only semi-sane workaround I know of is to use pip's `--only-binary` option to prevent any automatic builds at package installation time. Then you usually will also need your own package server for storing precompiled wheels for those third-party-dependencies that don't published compiled packages on pypi. This way you build the packages using some random tool version only once, so if it works the first time it will keep working. In other language ecosystems were the package management isn't built on a tower of shit, this problem doesn't exist in the first place.

duped 3y ago
Doesn't poetry have a solution for this?
nextlevelwizard 3y ago
>you are depending on just was silently downloading the latest version of some package you've never heard of, until it broke. How? Are you running `pip install` every time you run your pipeline? Why do you need to constantly be reinstalling your packages?