5 ms·
>Most people just cross their fingers and hope dependencies don't change Is there anything wrong with pip freeze > requirements.txt and then pip install -r req
by astonex 7y ago
>Most people just cross their fingers and hope dependencies don't change
Is there anything wrong with pip freeze > requirements.txt and then pip install -r requirements.txt ? This would install the exact versions
- zys5945 7y agoI think he is referring to indirect dependencies
- thaumasiotes 7y ago>>> Why isn't there any builtin way to automatically define a lock file pip isn't actually part of Python proper.
- jnwatson 7y agoIt isn't part of the Python executable, but it is part of the standard distribution.
- icebraining 7y agoNowadays, a regular installation of CPython lets you run "python -m pip". It's quite part of it.
- thaumasiotes 7y agoYou can do that with any library. You can issue Django commands by running `python -m django`; that doesn't change the fact that Django is a completely separate project from Python.
- voodoochicken 7y agoYes, those are caught when using pip freeze.
- zys5945 7y ago> Yes, those are caught when using pip freeze. No they are not. Pip freeze does not resolve transitive dependencies, nor does pip know what to do if your transitive dependencies conflict with each another.
- yzmtf2008 7y ago>Pip freeze does not resolve transitive dependencies How? Doesn't pip freeze literally list all packages that's installed in the current environment besides basic toolings such as setuptools (and you could even instruct it to list those as well)?
- geofft 7y ago> Pip freeze does not resolve transitive dependencies I don't think this is correct: $ python3 -m venv /tmp/v $ /tmp/v/bin/pip install flask [...] Collecting MarkupSafe>=0.23 (from Jinja2>=2.10.1->flask) [...] $ /tmp/v/bin/pip freeze | grep MarkupSafe MarkupSafe==1.1.1 > nor does pip know what to do if your transitive dependencies conflict with each another This is true, but because Python exposes all libraries in a single namespace at runtime, there isn't actually anything reasonable to do if they genuinely conflict. You can't have both, say, MarkupSafe 1.1.1 and MarkupSafe 1.1.0 in PYTHONPATH and expect them to be both accessible. There's no way in an import statement to say which one you want. However, it's notable that pip runs into trouble in cases where transitive dependencies don't genuinely conflict, too. See https://github.com/pypa/pip/issues/988 https://github.com/pypa/pip/issues/988 - this is a bug / acknowledged deficiency, and there is work in progress towards fixing it.
- marmaduke 7y ago> There's no way in an import statement to say which one you want. This would be fixable with a sys path hook, were pip so inclined
- geofft 7y agoIt would change the semantics of the language. You could also write a sys.path hook to interpret the remainder of the file as Ruby and not Python, were pip so inclined.... (Also it's not clear what those changed semantics would be.)
- sciyoshi 7y agoI've had a good experience with pip-tools (https://github.com/jazzband/pip-tools/ https://github.com/jazzband/pip-tools/) which takes a requirements.in with loosely-pinned dependencies and writes your requirements.txt with the exact versions including transitive dependencies.
- takeda 7y agoSame here, in my team we had immediate dependencies defined in setup.cfg when PR was merged, a pip-compile was run and generated requirements.txt and store it in central database (in our case it was consul because that was easiest to get without involving ops). pip-sync was then called to install it in given environment, any promotion from devint -> qa -> staging -> prod, was just copying the requirements.txt from environment earlier and calling pip-sync.
- AdamM12 7y agoTake my upvote. This has helped us a ton. So nice that it resolves dependencies. Only issue we're running into is that we don't use it to manage our dependencies for our internal packages (only using it at the application level). I've been advocating we change so that we simply read in the generated requirements.txt/requirements-dev.txt in setup.py
- erichurkman 7y agoLate to the party but `pip-tools` also has a flag for its `pip-compile` flag: `--generate-hashes`. It generates SHA256 hashes that `pip install` checks.
- LaGrange 7y agoIt catches way too much. IPython, black and the testing libraries are _not_ a part of my actual dependencies and shouldn't be installed in production. A good UI for a dependency manager at the very least distinguishes between dev and production context, and ideally lets me define custom contexts.
- BinaryIdiot 7y agoThis is what I've always done. Develop using a few dependencies, freeze, continue development with reproducible builds. It has always included the sub-dependencies in the list so, as far as I can tell, this works great for that case...
- marble-drink 7y agoThe only problem with that is it's hard to keep the dependencies up to date. Pip-tools solves this problem.
- yyhhsj0521 7y agoWhy do you want to update your dependencies if they work? Isn't the whole point of dependency management to avoid using different versions of dependencies than the ones they have been tested on?
- wtetzner 7y agoYou want to easily be able to update dependencies (and retest with them, of course) to ensure you’re getting security updates, for one.
- marble-drink 7y agoSecurity fixes, performance enhancements, new features. There are many reasons. But the point is you update in a controlled manner. You don't just push the latest version of everything out on to prod, but you also don't keep pushing the same version that worked a year ago.
- Chris2048 7y agoIf you have "foo==0.1" installed, and foo has the dep "bar~=0.2" (current vers of bar 0.2.1). Then bar releases version 0.2.2 So your deps want bar 0.2.1, but foo now wants bar 0.2.2 This breaks your pip install. EDIT: there are a few other gotchas (please respond to this post if you know of any more) e.g. from https://medium.com/knerd/the-nine-circles-of-python-dependency-hell-481d53e3e025 https://medium.com/knerd/the-nine-circles-of-python-dependen... "If two of your dependencies are demanding overlapping versions of a library, pip will not necessarily install a version of this library that satisfies both requirements" e.g. https://github.com/pypa/pip/issues/2775 https://github.com/pypa/pip/issues/2775
- Chris2048 7y agoAnother more obscure one: https://github.com/pandas-dev/pandas/issues/27206 https://github.com/pandas-dev/pandas/issues/27206 All of a sudden, a numpy release pulls in a new version for a pandas build (that incidentally breaks for py2) This without involving a "~=", but rather because pandas needs to build from source, and chooses the latest numpy build to do so.
- bpicolo 7y agoIt's not functional for library dependencies, where you still need to manage the setup.py instead. Only useful for end-user application dependencies.