3 ms·

The most frustrating thing is that pip doesn't make it easy to use more loose declared dependencies while freezing to actual concrete dependencies for deploymen

The most frustrating thing is that pip doesn't make it easy to use more loose declared dependencies while freezing to actual concrete dependencies for deployment. Everybody rolls their own. > Python libraries should not pin dependencies. _Applications_ can pin dependencies, including all recursive dependencies of their libraries. Is the pypi package awscli an application or a library? poetry is frustrating in that it doesn't allow you to override a library's declared requirements to break conflicts. They refuse to add support [1][2] for the feature too. awscli for example causes huge package conflict issues that make poetry unusable. It's almost impossible not to run into a requirement conflict with awscli if you're using a broad set of packages, even though awscli will operate happily with a more broad set of requirements than it declares. [1] https://github.com/python-poetry/poetry/issues/697 https://github.com/python-poetry/poetry/issues/697 [2] https://github.com/python-poetry/poetry/issues/697#issuecomment-744235786 https://github.com/python-poetry/poetry/issues/697#issuecomm...

anderskaseorg 6y ago
For this purpose, I’m defining a “library” as any PyPI package that you expect to be able to install alongside other PyPI packages. This includes some counterintuitive ones like mypy, which needs to extract types from packages in the same environment as the code it’s checking. The awscli documentation recommends installing it into its own virtualenv, in which case pinned dependencies may be reasonable. There are tools like pipx to automate that. Though in practice, there are reasons that installing applications into their own virtualenv might be inconvenient, inefficient, or impossible. And even when it’s possible, it still comes with the risk of missing security updates unless upstream is doing a really good job of staying on top of them. I don’t think that respecting declared dependency bounds is a Poetry bug. Pip respects them too (at least as of 20.3, which enables the new resolver by default: https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020 https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip...). If a package declares unhelpful bounds, the package should be fixed. (And yes, that means its maintainer might have to deal with some extra issues being filed—that’s part of the job.)
sagichmal 6y ago
> Is the pypi package awscli an application or a library? Hopefully a library! As hopefully the AWS command-line interface is maintained and distributed separately from any SDK that powers it...
- X-Istence 6y ago
  boto core/boto/boto 3 are the libraries, which awscli drives.
orf 6y ago
Why on earth would you ever add awscli as a dependency? That makes very little sense. It’s an application (that is no longer distributed via pypi). You should use boto3

anderskaseorg 6y ago

For this purpose, I’m defining a “library” as any PyPI package that you expect to be able to install alongside other PyPI packages. This includes some counterintuitive ones like mypy, which needs to extract types from packages in the same environment as the code it’s checking. The awscli documentation recommends installing it into its own virtualenv, in which case pinned dependencies may be reasonable. There are tools like pipx to automate that. Though in practice, there are reasons that installing applications into their own virtualenv might be inconvenient, inefficient, or impossible. And even when it’s possible, it still comes with the risk of missing security updates unless upstream is doing a really good job of staying on top of them. I don’t think that respecting declared dependency bounds is a Poetry bug. Pip respects them too (at least as of 20.3, which enables the new resolver by default: https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020 https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip...). If a package declares unhelpful bounds, the package should be fixed. (And yes, that means its maintainer might have to deal with some extra issues being filed—that’s part of the job.)

sagichmal 6y ago

> Is the pypi package awscli an application or a library? Hopefully a library! As hopefully the AWS command-line interface is maintained and distributed separately from any SDK that powers it...

X-Istence 6y ago
boto core/boto/boto 3 are the libraries, which awscli drives.

orf 6y ago

Why on earth would you ever add awscli as a dependency? That makes very little sense. It’s an application (that is no longer distributed via pypi). You should use boto3