4 ms·

I've pretty much settled on poetry for most things at this point. It still has a ton of rough edges like error messages still closer to a stack trace than some

I've pretty much settled on poetry for most things at this point. It still has a ton of rough edges like error messages still closer to a stack trace than something actionable and it's slow as hell resolving but it does the job 90 percent of the time without issue. Also running everything in docker now anyway which side steps some other problems.

imglorp 4y ago
That's where we are as well, but yes the problems you didn't sidestep with poetry+docker are still there: the pypi fecosystem out there does not stop quantum fluctuations just because you want it to. If you pin too much you're not getting security fixes and if you don't pin enough, you get bitten every time a tertiary dep changes that you never asked for. Oh yeah and there are now trojans upstream almost daily.
- tempest_ 4y ago
  Yeah but that trade off is entirely separate. We trade security for speed(or "velocity" if you want to be jargon about it). I just pin everything and go through my projects every couple weeks and bump the deps (unless some really big CVE hits the news).
- gen220 4y ago
  Does any FOSS package ecosystem have a good solution to this particular set of problems? I know Go has historically done something interesting with selecting the "minimum viable version number" for each dependency, but that's the only relevant idea that comes to mind. With PyPi, at least it's relatively trivial to self-host a pypi instance and self-manage dependency updates. In the Python ecosystem (vs e.g. Node), at least the total set of first+second-order dependencies for most normal projects is quite small (e.g. medium double digits to low triple digits). It doesn't feel too painful to manage that magnitude of upgrades on a monthly or quarterly basis.
  - lmm 4y ago
    In the JVM ecosystem you get the version your immediate dependency was built/released against, transitive dependencies are never silently upgraded (though if you want to upgrade then it's one command). IME that's a much better default; it does mean that when a security issue is found in a low-level library you get everyone doing a bunch of releases that just bump their dependencies, but the consistent behaviour is very much worth it.
odiroot 4y ago
Poetry still doesn't seem to support PEP621, instead requiring its custom, vendor-specific shape of pyproject.toml.
- gen220 4y ago
  Have they spoken about why they don't support it? Or plans to support it in the future? It seems like the shortest path to a universal package management tool for Python is to add this capability to Poetry, rather than to build something new.
  - thetic 4y ago
    https://github.com/python-poetry/roadmap/issues/3 https://github.com/python-poetry/roadmap/issues/3
npage97 4y ago
For anyone curious, the reasoning for the slowness is briefly described here: https://python-poetry.org/docs/faq/#why-is-the-dependency-resolution-process-slow https://python-poetry.org/docs/faq/#why-is-the-dependency-re...

imglorp 4y ago

That's where we are as well, but yes the problems you didn't sidestep with poetry+docker are still there: the pypi fecosystem out there does not stop quantum fluctuations just because you want it to. If you pin too much you're not getting security fixes and if you don't pin enough, you get bitten every time a tertiary dep changes that you never asked for. Oh yeah and there are now trojans upstream almost daily.

tempest_ 4y ago
Yeah but that trade off is entirely separate. We trade security for speed(or "velocity" if you want to be jargon about it). I just pin everything and go through my projects every couple weeks and bump the deps (unless some really big CVE hits the news).
gen220 4y ago
Does any FOSS package ecosystem have a good solution to this particular set of problems? I know Go has historically done something interesting with selecting the "minimum viable version number" for each dependency, but that's the only relevant idea that comes to mind. With PyPi, at least it's relatively trivial to self-host a pypi instance and self-manage dependency updates. In the Python ecosystem (vs e.g. Node), at least the total set of first+second-order dependencies for most normal projects is quite small (e.g. medium double digits to low triple digits). It doesn't feel too painful to manage that magnitude of upgrades on a monthly or quarterly basis.
- lmm 4y ago
  In the JVM ecosystem you get the version your immediate dependency was built/released against, transitive dependencies are never silently upgraded (though if you want to upgrade then it's one command). IME that's a much better default; it does mean that when a security issue is found in a low-level library you get everyone doing a bunch of releases that just bump their dependencies, but the consistent behaviour is very much worth it.

odiroot 4y ago

Poetry still doesn't seem to support PEP621, instead requiring its custom, vendor-specific shape of pyproject.toml.

gen220 4y ago
Have they spoken about why they don't support it? Or plans to support it in the future? It seems like the shortest path to a universal package management tool for Python is to add this capability to Poetry, rather than to build something new.
- thetic 4y ago
  https://github.com/python-poetry/roadmap/issues/3 https://github.com/python-poetry/roadmap/issues/3

npage97 4y ago

For anyone curious, the reasoning for the slowness is briefly described here: https://python-poetry.org/docs/faq/#why-is-the-dependency-resolution-process-slow https://python-poetry.org/docs/faq/#why-is-the-dependency-re...