4 ms·
I 100% agree on this. But it's been pretty hilarious on how obstinate tool makers are about changing to Python's new pyproject.toml [1] Python packaging is a b
by epistasis 6y ago
I 100% agree on this. But it's been pretty hilarious on how obstinate tool makers are about changing to Python's new pyproject.toml [1]
Python packaging is a bit of a mess already, so when I recently started a new small package I wanted to choose tooling that would not clutter, but a fair amount of tool makers were reluctant to allow code into their repo that would use the unified toml rather than a ton of separate file.
.config is probably a better solution than a single top level toml, IMHO, but far more important is doing something unified rather than continuing the pollution of the top-level namespace, which merely obscures the project structure.
[1] https://snarky.ca/what-the-heck-is-pyproject-toml/ https://snarky.ca/what-the-heck-is-pyproject-toml/
- user5994461 6y agoPython is unified on a requirements.txt file with pip, to manage dependencies.
- woodruffw 6y agoI don’t think this is true. I’ve personally seen the following (and I’m positive there are others): 1. setup.py with self contained dependencies (this is my general preference) 2. setup.py loading from requirements.txt 3. Standalone requirements.txt 4. requirements.txt generated from requirements.in (or similar) 5. pyproject.toml + Poetry with a dependencies section
- wwright 6y agorequirements.txt has a lot of deficiencies for package management (compare to the various features Yarn or Cargo offer). The biggest weakness is transitive dependency locking: even if you specify the exact version of the dependencies you want, Pip can still resolve to a different version of their dependencies, and even then there’s no verification that the content matches what is expected beyond the version number (other tools use a content checksum). This can cause a long tail of reproducibility and maintenance issues. It’s also a tiny subset of what the parent describes.
- bpicolo 6y agoIt also can't separate dev vs test vs prod dependencies, for example. Maintaining separate files for those is not exactly a convenient solution
- Sir_Substance 6y agoWhen I first set up my requirements.txt's, I usually set up a venv, install the things that I need and then run "pip freeze" to get a list of all deps including transitive deps, and put them all in the requirements.txts. I do sometimes feel that people are making rube goldberg machines out of their package management in an attempt to avoid just writing down all their deps.
- gruez 6y ago>The biggest weakness is transitive dependency locking: even if you specify the exact version of the dependencies you want, Pip can still resolve to a different version of their dependencies Isn't that what pip freeze is for?
- lmm 6y agoThe trouble there is that if you freeze you can't unfreeze. A project needs the equivalent of both gemfile and gemfile.lock (unless you're using a system like maven that got things right the first time), but there's no standard way of having both of them in Python.
- HALtheWise 6y agoI strongly recommend https://github.com/jazzband/pip-tools https://github.com/jazzband/pip-tools to solve this. It provides a simple script to take a requirements file and "compile" a full specification of all transitive dependencies. You check both files into the repo, point pip at the generated file, and manually modify the other one. It means you often don't need to pin requirements manually at all, and the versions will be explicitly updated whenever you choose to recompile your requirements.
- jniedrauer 6y agoThis is a big oversimplification. There are often multiple setup.txt files for different environments, virtualenv setup scripts, tox files, several different build config files, manifest files, and so on. Python probably has more magic config files than C#, which is quite an accomplishment.
- klyrs 6y agoNot "a" requirements.txt file, sadly. I've seen packages with different requirements for building, installing, and testing. All with their own damned txt file. Oh! And don't forget the duplication in setup.py -- you'll want to put your requirements there, too.
- 1f60c 6y agoAs far as I know, Pip doesn't have the notion of dev dependencies, so you need at least two requirements[.dev].txt files.
- dragonwriter 6y ago> As far as I know, Python doesn't have the notion of dev dependencies Python doesn't have the notion of dependencies. Pip doesn't have dev dependencies, poetry does.
- 1f60c 6y agoD’oh! Thanks. :)
- pas 6y agoThat's (the deps, not the dev deps) likely to change with pip 20.3! All hail the dependency resolver. It's already available behind a feature flag. https://pythoninsider.blogspot.com/2020/07/upgrade-pip-20-2-changes-20-3.html https://pythoninsider.blogspot.com/2020/07/upgrade-pip-20-2-...
- deleted 6y ago[deleted]
- jjoonathan 6y agoBut there are usually several versions of python with several pips, possibly in combination with anaconda with multiple environments, a combination of named and rolling releases, and a SAT solver that's always slow but occasionally spontaneously decides to become extra slow and extend your build time by hours. Oh and I forgot about setup.py. That too.
- dfee 6y agoI tried to downvote this, but I think it’s already downvoted to the max. There is so much wrong with this pattern. Please use something like Poetry to manage deps.
- paulie_a 6y agoI never have a unified requirements file.
- airstrike 6y agoThat link is a really good read. Better than TFA, tbh. And also particularly well written > An interesting side-effect of PEP 518 trying to introduce a standard file that all projects should (eventually) have is that non-build development tools realized they now had a file where they could put their own configuration. I say this is interesting because originally PEP 518 disallowed this, but people chose to ignore this part of the PEP xD We eventually updated the PEP to allow for this use-case since it became obvious people liked the idea of centralizing configuration data in a single file. Hilarious and indeed interesting
- franciscop 6y agoThis happens often in Node.js where you can put the configuration of some tools (e.g. babel, jest) inside package.json
- kdeldycke 6y agoI love pyproject.toml. I managed to get rid of setup.py, requirements.txt, requirements-dev.txt and MANIFEST.in, and I'm not far from having no setup.cfg soon. All others stuff (CI/CD workflows, linters, formatters, code of conduct, funding, issue and PR automation, dependency bots, ...) moved to .github. Now I have a clean and tidy Python project with a tiny 12-items root directory: https://github.com/kdeldycke/meta-package-manager https://github.com/kdeldycke/meta-package-manager