4 ms·

there are way too many python dep/env managers/things pipenv pyenv mkvirtualenv virtualenv pipsi venv pew conda virtualenvwrapper i'm sure i'm forgett

there are way too many python dep/env managers/things pipenv pyenv mkvirtualenv virtualenv pipsi venv pew conda virtualenvwrapper i'm sure i'm forgetting like 5. this is like https://xkcd.com/927/ https://xkcd.com/927/ for the record i use pyenv and virtualenv (although playing with ML i'm using conda)

yeukhon 9y ago
As a Python developer, I just stay with pip and virtualenv, but I share the smh/wtf sentiment here. The data science folks enjoy conda but I don't see that being useful to me anywhere else. I feel like every other year someone will invent a new pip env. pyvenv (see comment section, edited) is now deprecated and venv is recommended (shipped as part of Python 3.6 installation) is another confusion. Lest not forget the confusion of distuilts and setuptools is like argparse vs optparse in the past (which are both horrible). The experience of using pip and pypi (now Warehouse) is much better than that of Ruby and of NodeJS, but these "2-in-1" tools are just ridiculously "creative". As always, pro-tip: consider using the following to ensure environment is loaded properly when you are deploying production /full_path/env/bin/python myapp.py --workers=3 over source /full_path/env/activate && python myapp.py --workers=3 The latter is fine when you are doing local development in your terminals. If you go on #python IRC channel, every year a group of helpers will collectively recommend one of the above and then perhaps a different one the following year, so please do yourself a favor, just stick to pip and virtualenv.
- Cyph0n 9y ago
  Conda is especially useful if you want to use scientific packages on Windows.
  - SatvikBeri 9y ago
    Yeah, I use pip + virtualenv on most machines (and have for years), but conda is the only one that actually seems to "just work" on Windows.
  - yeukhon 9y ago
    Good point, thank you. I don't do much on Windows anymore. But a little search of VS Studio (I know there are some core envelopers working for MSFT) yields this: https://stackoverflow.com/questions/15185827/can-pip-be-used-with-python-tools-in-visual-studio https://stackoverflow.com/questions/15185827/can-pip-be-used... Probably exciting for VS users.
- jxm262 9y ago
  Apologies if I'm missing something obvious (relatively new to Python), but what's the advantage of loading it the 1st way? I'm currently using conda and do something similar to you're 2nd example `source activate conda-env && python myapp.py`
  - yeukhon 9y ago
    I'd like to hear what people think as well, so everyone please fire away. The 1st option is very explicit and doesn't modify your environment. The problem I have with the latter option is not just messing with my environment variable (bulk of it is setting PATH), but what will be changed in future releases. I don't have time to read the activate script (btw, there is one activate script for sh/bash/zsh, csh, fish, and bat, but if you are none of them, which likely a very corner case, you are on your own). Using the 1st method, I am certain the shell isn't being modified and loaded with stuff I don't need, and the command and arguments are explicit (when I look at top, or when I do strace). Generally, you can continue to do #2; I use it when I am doing development, like having multiple terminals up and just source into the environment, then run pytest instead of the full path to pytest. But I highly recommend #1 when you are running your code in production (webapp or not). I am not too familiar with conda so I will defer that to the experts).
- lettuce 9y ago
  > pyenv is now deprecated and venv is recommended Isn't it `pyvenv` that is deprecated? `pyvenv` != `pyenv`
  - yeukhon 9y ago
    Yes, you are right. See, that's the other (#%#$^#$%#$ - excuse my Chinese) confusion. Such conflicting name pyvenv and pyenv, someone really could have picked a better name, whoever came later. Corrected in my post.
    - wyclif 9y ago
      Thanks for saying this! I just spent the last 10 minutes confused by that, trying to figure out what was going on here and what I actually have installed now: ⟩ pyenv --version zsh: correct 'pyenv' to 'pyvenv' [nyae]?
- unpythonic 9y ago
  If you want to not infect your current shell with the new environment, you can just run it in a subshell by using parentheses: $ ( . /full_path/env/bin/activate && python myapp.py --workers=3 ) This works for anything that you'd like a temporary shell for: $ pwd /home/me $ ( cd /tmp; pwd ) /tmp $ pwd /home/me
- jordic 9y ago
  python3.6 -m venv . && source bin/activate (this builds around the idea every project is a python package itself). pip install -e .
sevensor 9y ago
It's also super easy to build Python from source and manage the paths yourself, in case the magic tools fail to deliver the desired Python version.
user5994461 9y ago
There is only pip and virtualenv. They go together. Virtualenv creates an application directory. Pip install the required packages into it.
meribold 9y ago
More than 5: I recently listed all programs for managing virtual environments I could find in a blog post [1] because I find it pretty confusing how many there are. [1]: https://meribold.github.io/virtual-environments-9487 https://meribold.github.io/virtual-environments-9487
hathawsh 9y ago
I use Buildout: http://docs.buildout.org/en/latest/ http://docs.buildout.org/en/latest/ Buildout is much more than a Python package manager. It runs pluggable recipes, where building/installing a Python package is only one of the many available recipes. It was invented at a company where I once worked. It replaced huge piles of complicated Makefiles. Buildout is obviously less popular than other solutions, but I think it manages complexity quite well and saves me a lot of time compared with other ways I could assemble Python software.
masklinn 9y ago
Well pyenv is to manage multiple versions of python, virtualenv is to create virtual environment, venv is virtualenv added to the stdlib, you forgot pip, virtualenvwrapper and mkvirtualenv are management tools for venvs and pipenv is also that but integrating pip. There's pyenv, pip and virtualenv/venv (same thing) and then there's a bunch of tools to make them more implicit or ergonomic (if you like the way these tools work more than the base). So really there's not a multitude of competing standards, there's 2/3 complementary standards and then people building their own tools atop that, not entirely unlike the multitudes of Jabber or Twitter clients we used to have. Things were worse before virtualenv rose to prominence.
brettcannon 9y ago
Please realize that the concept of virtual environments in Python predate things like npm, so there are lessons that were learned later on that no one knew about. Also realize that the things you list all layer on top of each other so you're listing lower-level libraries next to higher-level ones. E.g. pipenv <- pew, pyenv <- virtualenv/venv, so that list could be likened to complaining that Python, C, assembly, and CPU microcode is "too many". Sure, you could argue they are usable independent of each other, but you aren't about to say that there are too many pieces of tech there and we should stop and sending raw electricity to the CPU. I think that list could legitimately be cut down to pipenv and conda (maybe pipsi, but that's just for installing CLI tools and isn't for development). Everything else is lower-level than what most people will need for app development.

yeukhon 9y ago

As a Python developer, I just stay with pip and virtualenv, but I share the smh/wtf sentiment here. The data science folks enjoy conda but I don't see that being useful to me anywhere else. I feel like every other year someone will invent a new pip env. pyvenv (see comment section, edited) is now deprecated and venv is recommended (shipped as part of Python 3.6 installation) is another confusion. Lest not forget the confusion of distuilts and setuptools is like argparse vs optparse in the past (which are both horrible). The experience of using pip and pypi (now Warehouse) is much better than that of Ruby and of NodeJS, but these "2-in-1" tools are just ridiculously "creative". As always, pro-tip: consider using the following to ensure environment is loaded properly when you are deploying production /full_path/env/bin/python myapp.py --workers=3 over source /full_path/env/activate && python myapp.py --workers=3 The latter is fine when you are doing local development in your terminals. If you go on #python IRC channel, every year a group of helpers will collectively recommend one of the above and then perhaps a different one the following year, so please do yourself a favor, just stick to pip and virtualenv.

Cyph0n 9y ago
Conda is especially useful if you want to use scientific packages on Windows.
- SatvikBeri 9y ago
  Yeah, I use pip + virtualenv on most machines (and have for years), but conda is the only one that actually seems to "just work" on Windows.
- yeukhon 9y ago
  Good point, thank you. I don't do much on Windows anymore. But a little search of VS Studio (I know there are some core envelopers working for MSFT) yields this: https://stackoverflow.com/questions/15185827/can-pip-be-used-with-python-tools-in-visual-studio https://stackoverflow.com/questions/15185827/can-pip-be-used... Probably exciting for VS users.
jxm262 9y ago
Apologies if I'm missing something obvious (relatively new to Python), but what's the advantage of loading it the 1st way? I'm currently using conda and do something similar to you're 2nd example `source activate conda-env && python myapp.py`
- yeukhon 9y ago
  I'd like to hear what people think as well, so everyone please fire away. The 1st option is very explicit and doesn't modify your environment. The problem I have with the latter option is not just messing with my environment variable (bulk of it is setting PATH), but what will be changed in future releases. I don't have time to read the activate script (btw, there is one activate script for sh/bash/zsh, csh, fish, and bat, but if you are none of them, which likely a very corner case, you are on your own). Using the 1st method, I am certain the shell isn't being modified and loaded with stuff I don't need, and the command and arguments are explicit (when I look at top, or when I do strace). Generally, you can continue to do #2; I use it when I am doing development, like having multiple terminals up and just source into the environment, then run pytest instead of the full path to pytest. But I highly recommend #1 when you are running your code in production (webapp or not). I am not too familiar with conda so I will defer that to the experts).
lettuce 9y ago
> pyenv is now deprecated and venv is recommended Isn't it `pyvenv` that is deprecated? `pyvenv` != `pyenv`
- yeukhon 9y ago
  Yes, you are right. See, that's the other (#%#$^#$%#$ - excuse my Chinese) confusion. Such conflicting name pyvenv and pyenv, someone really could have picked a better name, whoever came later. Corrected in my post.
  - wyclif 9y ago
    Thanks for saying this! I just spent the last 10 minutes confused by that, trying to figure out what was going on here and what I actually have installed now: ⟩ pyenv --version zsh: correct 'pyenv' to 'pyvenv' [nyae]?
unpythonic 9y ago
If you want to not infect your current shell with the new environment, you can just run it in a subshell by using parentheses: $ ( . /full_path/env/bin/activate && python myapp.py --workers=3 ) This works for anything that you'd like a temporary shell for: $ pwd /home/me $ ( cd /tmp; pwd ) /tmp $ pwd /home/me
jordic 9y ago
python3.6 -m venv . && source bin/activate (this builds around the idea every project is a python package itself). pip install -e .

sevensor 9y ago

It's also super easy to build Python from source and manage the paths yourself, in case the magic tools fail to deliver the desired Python version.

user5994461 9y ago

There is only pip and virtualenv. They go together. Virtualenv creates an application directory. Pip install the required packages into it.

meribold 9y ago

More than 5: I recently listed all programs for managing virtual environments I could find in a blog post [1] because I find it pretty confusing how many there are. [1]: https://meribold.github.io/virtual-environments-9487 https://meribold.github.io/virtual-environments-9487

hathawsh 9y ago

I use Buildout: http://docs.buildout.org/en/latest/ http://docs.buildout.org/en/latest/ Buildout is much more than a Python package manager. It runs pluggable recipes, where building/installing a Python package is only one of the many available recipes. It was invented at a company where I once worked. It replaced huge piles of complicated Makefiles. Buildout is obviously less popular than other solutions, but I think it manages complexity quite well and saves me a lot of time compared with other ways I could assemble Python software.

masklinn 9y ago

Well pyenv is to manage multiple versions of python, virtualenv is to create virtual environment, venv is virtualenv added to the stdlib, you forgot pip, virtualenvwrapper and mkvirtualenv are management tools for venvs and pipenv is also that but integrating pip. There's pyenv, pip and virtualenv/venv (same thing) and then there's a bunch of tools to make them more implicit or ergonomic (if you like the way these tools work more than the base). So really there's not a multitude of competing standards, there's 2/3 complementary standards and then people building their own tools atop that, not entirely unlike the multitudes of Jabber or Twitter clients we used to have. Things were worse before virtualenv rose to prominence.

brettcannon 9y ago

Please realize that the concept of virtual environments in Python predate things like npm, so there are lessons that were learned later on that no one knew about. Also realize that the things you list all layer on top of each other so you're listing lower-level libraries next to higher-level ones. E.g. pipenv <- pew, pyenv <- virtualenv/venv, so that list could be likened to complaining that Python, C, assembly, and CPU microcode is "too many". Sure, you could argue they are usable independent of each other, but you aren't about to say that there are too many pieces of tech there and we should stop and sending raw electricity to the CPU. I think that list could legitimately be cut down to pipenv and conda (maybe pipsi, but that's just for installing CLI tools and isn't for development). Everything else is lower-level than what most people will need for app development.