4 ms·

> I had a friend who decided to not use uv, because the first time he used it, it was on a 15 years old codebase that had just been migrated to Python 3. It was

by belval 2y ago

> I had a friend who decided to not use uv, because the first time he used it, it was on a 15 years old codebase that had just been migrated to Python 3. It was standing on a pile of never cleaned up pip freeze exports, and uv could not make it work. This is my only gripe with uv, despite how the author decided to depict it, this really turns into a headache fast as soon as you have ~4-5 in-house packages. I don't think it's that bad that uv is so unforgiving in those case because it leads to better overall project quality/cohesion, but I wish there was a way to more progressively onboard and downgrade minor version mismatch to warnings.

code_biologist 2y ago
Did they run into a hard blocker, or was it just that using version overrides was possible but painful? I started looking seriously at uv/pdm once poetry made it entirely clear they didn't intend to support version overrides [1]. uv's support for overrides seems serviceable if unsophisticated [2][3]. [1] https://github.com/python-poetry/poetry/issues/697 https://github.com/python-poetry/poetry/issues/697 [2] https://docs.astral.sh/uv/concepts/resolution/#dependency-overrides https://docs.astral.sh/uv/concepts/resolution/#dependency-ov... [3] https://docs.astral.sh/uv/reference/settings/#override-dependencies https://docs.astral.sh/uv/reference/settings/#override-depen...
- aaravchen 2y ago
  The linked poetry Issue is pretty understandable why they aren't going to support it. I've honestly never heard of any dependency resolver that allows you to dynamically inject an override of a package's built in specification for an indirect dependency. Point blank, that's a packaging failure and the solution is, and always has been, to immediately yank the offending package. That said, the Python case has pretty limited ability to specify dependency package versions, which makes it nigh impossible to handle it downstream by blocklisting specific versions from an otherwise contiguous range. Take for example the werkzeug package that released a breaking API regression in a patch release version. It didn't affect everyone, but notably did affect certain Flask(?) use cases that used werkzeug as a dependency. In a sane system, either werkzeug immediately removes the last released version as buggy (and optionally re-releases it as a non-backwards compatible SemVer change), or everyone starts looks for an alternative to non-compliant werkzeug. Pragmatically though, Python dependency specification syntax should have a way for Flask to specify, in a patch release of its own, that werkzeug up to the next minor version, _but excluding a specific range of patch versions_, is a dependency. Allowing them to monkey patch the problem in the short term. It should never be on the end user to be specifying overrides of indirect dependency specifications at the top level though, which is what was requested from the poetry tool.
  - lmm 2y ago
    > I've honestly never heard of any dependency resolver that allows you to dynamically inject an override of a package's built in specification for an indirect dependency. You can do it with [patch] in cargo (I think), or .exclude in SBT. In Maven you can use <dependencyManagement>. In fact I can't think of a package manager that doesn't support it, it's something I'd always expect to be possible. > Point blank, that's a packaging failure and the solution is, and always has been, to immediately yank the offending package. Be that as it may, PyPi won't. > It should never be on the end user to be specifying overrides of indirect dependency specifications at the top level though It "shouldn't", but sometimes the user will find themselves in that situation. The only choice is whether you give them the tools to work around it or you don't.
  - code_biologist 2y ago
    I've honestly never heard of any dependency resolver that allows you to dynamically inject an override of a package's built in specification for an indirect dependency. npm and yarn both let you do it. PDM and uv think about it differently, but both allow overrides. It should never be on the end user to be specifying overrides of indirect dependency specifications at the top level though, which is what was requested from the poetry tool. I'm jealous of your upstreams. I just want to use Django package XYZ that says it's only compatible with Django 3.X on Django 4. Works just fine, but poetry won't let it happen. Upstream seems like they might literally be dead in some cases, with an unmerged unanswered PR open for years. In other cases a PR was merged but no new PyPI release was ever made because I allowed for more liberal requirements for a 0.7.X release last made in 2019 and they're on version 4.X or whatever these days. On one decade old application I have a half dozen forks of old packages with only alterations to dependency specifications specifically to please poetry. It's really annoying as opposed to just being able to say "I know better than what this package says" like in npm and yarn. This is exactly what half the comments in poetry's "please allow overrides" issue are saying.
    - zahlman 2y ago
      >I just want to use Django package XYZ that says it's only compatible with Django 3.X on Django 4. Indeed. One of the biggest recurring themes I've seen in Python packaging discussion is that package metadata can't be updated after the fact. When you publish something that depends on the just-released foolib N, you don't know if it will be compatible with foolib N+1 (you might not even be completely sure it will work with N.0.1) because it doesn't even exist yet so you can't possibly test it. In other languages, where the environment can contain multiple versions of the same library, it's common to assume it won't. For Python, there are lots of good reasons to assume it will (https://iscinumpy.dev/post/bound-version-constraints/ https://iscinumpy.dev/post/bound-version-constraints/), but people will blame you when you're wrong and the fix isn't straightforward. (AIUI, the best you can do is make a .post1 release with the updated metadata, and "yank" the existing release.)
halfcat 2y ago
> this really turns into a headache fast as soon as you have ~4-5 in-house packages What’s a typical or better way of handling in-house packages?
- belval 2y ago
  By default pip will give you a warning that package X requires package Y ==1.0 but you have 1.1 which is incompatible instead of just failing. That's the feature that I'd like to have, basically a "--just-warn" flag or a way to set a package as "version does not matter". > What’s a typical or better way of handling in-house packages? Fixing your dependencies properly, but on some older codebases that pull also old dependencies this be a headache. For example "Pillow" a Python image library is a dependency in just about everything that manipulates images. This means that one package might have >=9.6<9.7, some package will have ==9.8 and another will have >=10<11. In practice it never matters and any of those version would work but you have a "version deadlock" and now you need to bump the version in packages that you may not actually own. Having some override of "this project uses Pillow==10, if some package ask for something else, ignore it" is something that pip does that uv doesn't.

by belval 2y ago

code_biologist 2y ago

Did they run into a hard blocker, or was it just that using version overrides was possible but painful? I started looking seriously at uv/pdm once poetry made it entirely clear they didn't intend to support version overrides [1]. uv's support for overrides seems serviceable if unsophisticated [2][3]. [1] https://github.com/python-poetry/poetry/issues/697 https://github.com/python-poetry/poetry/issues/697 [2] https://docs.astral.sh/uv/concepts/resolution/#dependency-overrides https://docs.astral.sh/uv/concepts/resolution/#dependency-ov... [3] https://docs.astral.sh/uv/reference/settings/#override-dependencies https://docs.astral.sh/uv/reference/settings/#override-depen...

aaravchen 2y ago
The linked poetry Issue is pretty understandable why they aren't going to support it. I've honestly never heard of any dependency resolver that allows you to dynamically inject an override of a package's built in specification for an indirect dependency. Point blank, that's a packaging failure and the solution is, and always has been, to immediately yank the offending package. That said, the Python case has pretty limited ability to specify dependency package versions, which makes it nigh impossible to handle it downstream by blocklisting specific versions from an otherwise contiguous range. Take for example the werkzeug package that released a breaking API regression in a patch release version. It didn't affect everyone, but notably did affect certain Flask(?) use cases that used werkzeug as a dependency. In a sane system, either werkzeug immediately removes the last released version as buggy (and optionally re-releases it as a non-backwards compatible SemVer change), or everyone starts looks for an alternative to non-compliant werkzeug. Pragmatically though, Python dependency specification syntax should have a way for Flask to specify, in a patch release of its own, that werkzeug up to the next minor version, _but excluding a specific range of patch versions_, is a dependency. Allowing them to monkey patch the problem in the short term. It should never be on the end user to be specifying overrides of indirect dependency specifications at the top level though, which is what was requested from the poetry tool.
- lmm 2y ago
  > I've honestly never heard of any dependency resolver that allows you to dynamically inject an override of a package's built in specification for an indirect dependency. You can do it with [patch] in cargo (I think), or .exclude in SBT. In Maven you can use <dependencyManagement>. In fact I can't think of a package manager that doesn't support it, it's something I'd always expect to be possible. > Point blank, that's a packaging failure and the solution is, and always has been, to immediately yank the offending package. Be that as it may, PyPi won't. > It should never be on the end user to be specifying overrides of indirect dependency specifications at the top level though It "shouldn't", but sometimes the user will find themselves in that situation. The only choice is whether you give them the tools to work around it or you don't.
- code_biologist 2y ago
  I've honestly never heard of any dependency resolver that allows you to dynamically inject an override of a package's built in specification for an indirect dependency. npm and yarn both let you do it. PDM and uv think about it differently, but both allow overrides. It should never be on the end user to be specifying overrides of indirect dependency specifications at the top level though, which is what was requested from the poetry tool. I'm jealous of your upstreams. I just want to use Django package XYZ that says it's only compatible with Django 3.X on Django 4. Works just fine, but poetry won't let it happen. Upstream seems like they might literally be dead in some cases, with an unmerged unanswered PR open for years. In other cases a PR was merged but no new PyPI release was ever made because I allowed for more liberal requirements for a 0.7.X release last made in 2019 and they're on version 4.X or whatever these days. On one decade old application I have a half dozen forks of old packages with only alterations to dependency specifications specifically to please poetry. It's really annoying as opposed to just being able to say "I know better than what this package says" like in npm and yarn. This is exactly what half the comments in poetry's "please allow overrides" issue are saying.
  - zahlman 2y ago
    >I just want to use Django package XYZ that says it's only compatible with Django 3.X on Django 4. Indeed. One of the biggest recurring themes I've seen in Python packaging discussion is that package metadata can't be updated after the fact. When you publish something that depends on the just-released foolib N, you don't know if it will be compatible with foolib N+1 (you might not even be completely sure it will work with N.0.1) because it doesn't even exist yet so you can't possibly test it. In other languages, where the environment can contain multiple versions of the same library, it's common to assume it won't. For Python, there are lots of good reasons to assume it will (https://iscinumpy.dev/post/bound-version-constraints/ https://iscinumpy.dev/post/bound-version-constraints/), but people will blame you when you're wrong and the fix isn't straightforward. (AIUI, the best you can do is make a .post1 release with the updated metadata, and "yank" the existing release.)

halfcat 2y ago

> this really turns into a headache fast as soon as you have ~4-5 in-house packages What’s a typical or better way of handling in-house packages?

belval 2y ago
By default pip will give you a warning that package X requires package Y ==1.0 but you have 1.1 which is incompatible instead of just failing. That's the feature that I'd like to have, basically a "--just-warn" flag or a way to set a package as "version does not matter". > What’s a typical or better way of handling in-house packages? Fixing your dependencies properly, but on some older codebases that pull also old dependencies this be a headache. For example "Pillow" a Python image library is a dependency in just about everything that manipulates images. This means that one package might have >=9.6<9.7, some package will have ==9.8 and another will have >=10<11. In practice it never matters and any of those version would work but you have a "version deadlock" and now you need to bump the version in packages that you may not actually own. Having some override of "this project uses Pillow==10, if some package ask for something else, ignore it" is something that pip does that uv doesn't.