4 ms·

Have a look at pipenv - [https://docs.pipenv.org/ https://docs.pipenv.org/], it provides deterministic builds and sane package management for python and is the

Have a look at pipenv - [https://docs.pipenv.org/ https://docs.pipenv.org/], it provides deterministic builds and sane package management for python and is the officially recommended Python packaging tool now. If dependency management is your major problem, Docker might be an overkill. P.S. -- Even if you are on Windows, whole SciPy ecosystem can be easily installed with pipenv, no need of conda for that now.

j2kun 9y ago
This looks pretty, but I've never personally had trouble with `pip freeze > requirements.txt` What benefits does pipenv have beyond storing the hashes as well? In what way is it more sane? For example, does it get around the issues with, e.g., matplotlib being installed in a virtualenv?
- abhirag 9y ago
  I assume you are referring to the issues related to installation of external GUI frameworks in virtual environments mentioned here -- [https://matplotlib.org/faq/virtualenv_faq.html https://matplotlib.org/faq/virtualenv_faq.html], I am afraid pipenv doesn't solve that any better than virtualenv’s --system-site-packages option -- [https://docs.pipenv.org/advanced/#working-with-platform-provided-python-components https://docs.pipenv.org/advanced/#working-with-platform-prov...]. If you are happy with your pip and venv workflow you can happily ignore pipenv, I personally prefer it because: - It combines all the features of pip and venv with a better UI(in my opinion). - I can specify my environments in more detail using Pipfile [https://docs.pipenv.org/advanced/#specifying-basically-anything https://docs.pipenv.org/advanced/#specifying-basically-anyth...] - It is integrated with pyenv [https://docs.pipenv.org/advanced/#automatic-python-installation https://docs.pipenv.org/advanced/#automatic-python-installat...] - The promise of deterministic builds
pwang 9y ago
Pipenv is only as good as the wheels that get installed. And the wheels are only as good as the assumption that your runtime/deployment environment matches the buildtime environment of the person who made those wheels. Oftentimes with ML and compute-intensive workloads, performance is an aspect of correctness. It is nontrivial to get the right accelerated libraries, and the right GPU libraries linking against the right Numpy version for the particular notebook you're running. If you are doing work just for yourself, or if you have total control over the deployment environment (and your collaborator's environments), then you might be able to get away with just using pip for these things.
- abhirag 9y ago
  Agreed :) when linking with native libraries conda definitely helps a lot. I was merely trying to point to the fact that after the recent release of SciPy we finally have binary wheels for Windows for the whole SciPy ecosystem.

j2kun 9y ago

This looks pretty, but I've never personally had trouble with `pip freeze > requirements.txt` What benefits does pipenv have beyond storing the hashes as well? In what way is it more sane? For example, does it get around the issues with, e.g., matplotlib being installed in a virtualenv?

abhirag 9y ago
I assume you are referring to the issues related to installation of external GUI frameworks in virtual environments mentioned here -- [https://matplotlib.org/faq/virtualenv_faq.html https://matplotlib.org/faq/virtualenv_faq.html], I am afraid pipenv doesn't solve that any better than virtualenv’s --system-site-packages option -- [https://docs.pipenv.org/advanced/#working-with-platform-provided-python-components https://docs.pipenv.org/advanced/#working-with-platform-prov...]. If you are happy with your pip and venv workflow you can happily ignore pipenv, I personally prefer it because: - It combines all the features of pip and venv with a better UI(in my opinion). - I can specify my environments in more detail using Pipfile [https://docs.pipenv.org/advanced/#specifying-basically-anything https://docs.pipenv.org/advanced/#specifying-basically-anyth...] - It is integrated with pyenv [https://docs.pipenv.org/advanced/#automatic-python-installation https://docs.pipenv.org/advanced/#automatic-python-installat...] - The promise of deterministic builds

pwang 9y ago

Pipenv is only as good as the wheels that get installed. And the wheels are only as good as the assumption that your runtime/deployment environment matches the buildtime environment of the person who made those wheels. Oftentimes with ML and compute-intensive workloads, performance is an aspect of correctness. It is nontrivial to get the right accelerated libraries, and the right GPU libraries linking against the right Numpy version for the particular notebook you're running. If you are doing work just for yourself, or if you have total control over the deployment environment (and your collaborator's environments), then you might be able to get away with just using pip for these things.

abhirag 9y ago
Agreed :) when linking with native libraries conda definitely helps a lot. I was merely trying to point to the fact that after the recent release of SciPy we finally have binary wheels for Windows for the whole SciPy ecosystem.