5 ms·

Python Virtual Environments – a Primer

115 points by mjhea0 11y ago

sk3w 11y ago
Note that pyvenv != pyenv, and the linked SO article appears relevant to the latter, not the former. Naming is hard.
- ubernostrum 11y ago
  Well, pyenv does have rather good virtualenv integration, and in fact I use that to make it easy to spin up a virtualenv with whatever Python version I happen to need at the moment.
- recursive 11y ago
  I was puzzling over that exact thing for maybe 15 minutes. I've never used either one. Neither is on my path currently, even though I just installed python 3.5 with that option ticked. I can find pyvenv.py in my file system. Perhaps I'm expected to add it to my system path manually? Maybe I'll figure this out another day.
beagle3 11y ago
I do not have a lot of experience, but I do have some, and in my experience miniconda is superior to virtual env.
- lqdc13 11y ago
  Some packages don't install correctly with *conda.
  - crucialfelix 11y ago
    you don't have to use conda to install. use pip as normal.
    - lqdc13 11y ago
      Just looked at it again. Seems like it's better than before. Will give it a try.
crucialfelix 11y ago
I've switched to using conda for managing all of my local python projects and tools. Its an easier, more flexible interface and it can switch both the virtualenv and the python version. So no problem going back and forth from py3 to 2 Just install miniconda (without all the data science packages) and then use pip as normal http://conda.pydata.org/docs/using/envs.html http://conda.pydata.org/docs/using/envs.html Autoswitching when you enter a directory: https://github.com/chdoig/conda-auto-env https://github.com/chdoig/conda-auto-env
- japhyr 11y ago
  How well do Django-based projects work with conda?
  - crucialfelix 11y ago
    I'm running that in Vagrant anyway, but it would work just fine. Conda is just a well done wrapper for switching python versions and virtualenvironments. You can also set ENVIRONMENT variables for each so that could be useful for specific django projects.
- p4wnc6 11y ago
  Most often you won't need to use pip, and `conda install ...` is preferable (for example, pretty much everything on PyPI is mirrored to anaconda.org, and if you ask them to support a well-known package that's not already available, it will be available extremely quickly, and chances are it's already available on some other anaconda.org channel anyway). You can use pip from within conda as well if you want to, and you can refer to environment-specific pips directly, such as `~/anaconda/envs/my_env_name/bin/pip install ...` to install for the environment my_env_name. conda can work with pip-styled requirements.txt files as well, and it's easy to either upload your own packages to anaconda.org, or create the conda build metadata (metadata.yaml) with instructions for how to get the package from PyPI if you prefer to only upload to PyPI. There is an abundance of well-written tutorials and documentation for these things.
  - epaulson 11y ago
njharman 11y ago
I've been very happy with vex https://pypi.python.org/pypi/vex https://pypi.python.org/pypi/vex
davexunit 11y ago
I currently use GNU Guix to make virtual environments [0] (sometimes using containers) for software written in any language, which I find much nicer than a Python specific solution. [0] http://www.gnu.org/software/guix/manual/html_node/Invoking-guix-environment.html http://www.gnu.org/software/guix/manual/html_node/Invoking-g...
- berdario 11y ago
  I also think that Nix/Guix are the way forward... Guix really needs an installer/Linux package, though Just yesterday I added a few tasks to the ansible playbook for my workstation to automate its installation: https://github.com/berdario/dotfiles/commit/4e92cdb00dbe135ffce46187efb78f4a201521c3 https://github.com/berdario/dotfiles/commit/4e92cdb00dbe135f... Small disclaimer: I'm also the author of Pew ( https://github.com/berdario/pew https://github.com/berdario/pew ), an alternative to virtualenvwrapper... and I'm working on a small program to try to automate further what pypi2nix/guix-import do to automatically package Python stuff for Nix/Guix... Until that'll be really easy, I think that whipping up a new pew virtualenv to try something out quickly is still the easiest thing (especially for people not used to Nix... I know: otherwise you can also use mutable virtualenvs on top of Nix)
STRiDEX 11y ago
Using virtualenvwrapper and the plugin included with oh my zsh virtualenviroments are easy, but it is good to know what's happening.
actionscripted 11y ago
We've moved to Docker Compose for local Python (Django) development. Oftentimes when you need package management (pip) and multiple environments (virtualenv) you likely also have several services necessary for your projects (Python, Postgres, Nginx, Celery, et al.) and it can be easier for your team to run `docker-compose up` and have the whole stack running in lieu of setting up services manually and then initializing a virtual environment. I much prefer taking the time to setup Docker Compose even for a simple project. I don't have to initialize a virtual environment, install packages, make sure the DB is running, sync with the DB, run the web server, etc. All of that work becomes automatic and I can focus on development instead sysadmin stuff.
ris 11y ago
Frankly I just use Nix, which is a little like having virtualenvs for your entire distribution and as such can handle non-python dependencies properly.
- pekk 11y ago
  Virtualenv isn't supposed to handle non-Python dependencies. Why use Nix instead of containers or VMs?
  - ris 11y ago
    "Virtualenv isn't supposed to handle non-Python dependencies." Making it kinda useless for anything like numpy or scipy. "Why use Nix instead of containers or VMs?" Well, why use virtualenvs instead of containers or VMs? Because it's lighter weight and more straightforward. And doesn't require you to swap out your whole working environment every time you want a slightly different package setup. Containers and VMs are a bad excuse for "I don't have a hope of getting the dependencies of my setup under control so I'm just going to ball them all up into this great big binary image." Nix packages are incredibly flexible and composable in comparison.
jaxondu 11y ago
I don't need the extra in virtualenvwrapper, just use the build-in venv module in Python 3: $ python3 -m venv myvenv $ source myvenv/bin/activate

115 points by mjhea0 11y ago

sk3w 11y ago

Note that pyvenv != pyenv, and the linked SO article appears relevant to the latter, not the former. Naming is hard.

ubernostrum 11y ago
Well, pyenv does have rather good virtualenv integration, and in fact I use that to make it easy to spin up a virtualenv with whatever Python version I happen to need at the moment.
recursive 11y ago
I was puzzling over that exact thing for maybe 15 minutes. I've never used either one. Neither is on my path currently, even though I just installed python 3.5 with that option ticked. I can find pyvenv.py in my file system. Perhaps I'm expected to add it to my system path manually? Maybe I'll figure this out another day.

beagle3 11y ago

I do not have a lot of experience, but I do have some, and in my experience miniconda is superior to virtual env.

lqdc13 11y ago
Some packages don't install correctly with *conda.
- crucialfelix 11y ago
  you don't have to use conda to install. use pip as normal.
  - lqdc13 11y ago
    Just looked at it again. Seems like it's better than before. Will give it a try.

crucialfelix 11y ago

I've switched to using conda for managing all of my local python projects and tools. Its an easier, more flexible interface and it can switch both the virtualenv and the python version. So no problem going back and forth from py3 to 2 Just install miniconda (without all the data science packages) and then use pip as normal http://conda.pydata.org/docs/using/envs.html http://conda.pydata.org/docs/using/envs.html Autoswitching when you enter a directory: https://github.com/chdoig/conda-auto-env https://github.com/chdoig/conda-auto-env

japhyr 11y ago
How well do Django-based projects work with conda?
- crucialfelix 11y ago
  I'm running that in Vagrant anyway, but it would work just fine. Conda is just a well done wrapper for switching python versions and virtualenvironments. You can also set ENVIRONMENT variables for each so that could be useful for specific django projects.
p4wnc6 11y ago
Most often you won't need to use pip, and `conda install ...` is preferable (for example, pretty much everything on PyPI is mirrored to anaconda.org, and if you ask them to support a well-known package that's not already available, it will be available extremely quickly, and chances are it's already available on some other anaconda.org channel anyway). You can use pip from within conda as well if you want to, and you can refer to environment-specific pips directly, such as `~/anaconda/envs/my_env_name/bin/pip install ...` to install for the environment my_env_name. conda can work with pip-styled requirements.txt files as well, and it's easy to either upload your own packages to anaconda.org, or create the conda build metadata (metadata.yaml) with instructions for how to get the package from PyPI if you prefer to only upload to PyPI. There is an abundance of well-written tutorials and documentation for these things.
- epaulson 11y ago

njharman 11y ago

I've been very happy with vex https://pypi.python.org/pypi/vex https://pypi.python.org/pypi/vex

davexunit 11y ago

I currently use GNU Guix to make virtual environments [0] (sometimes using containers) for software written in any language, which I find much nicer than a Python specific solution. [0] http://www.gnu.org/software/guix/manual/html_node/Invoking-guix-environment.html http://www.gnu.org/software/guix/manual/html_node/Invoking-g...

berdario 11y ago
I also think that Nix/Guix are the way forward... Guix really needs an installer/Linux package, though Just yesterday I added a few tasks to the ansible playbook for my workstation to automate its installation: https://github.com/berdario/dotfiles/commit/4e92cdb00dbe135ffce46187efb78f4a201521c3 https://github.com/berdario/dotfiles/commit/4e92cdb00dbe135f... Small disclaimer: I'm also the author of Pew ( https://github.com/berdario/pew https://github.com/berdario/pew ), an alternative to virtualenvwrapper... and I'm working on a small program to try to automate further what pypi2nix/guix-import do to automatically package Python stuff for Nix/Guix... Until that'll be really easy, I think that whipping up a new pew virtualenv to try something out quickly is still the easiest thing (especially for people not used to Nix... I know: otherwise you can also use mutable virtualenvs on top of Nix)

STRiDEX 11y ago

Using virtualenvwrapper and the plugin included with oh my zsh virtualenviroments are easy, but it is good to know what's happening.

actionscripted 11y ago

We've moved to Docker Compose for local Python (Django) development. Oftentimes when you need package management (pip) and multiple environments (virtualenv) you likely also have several services necessary for your projects (Python, Postgres, Nginx, Celery, et al.) and it can be easier for your team to run `docker-compose up` and have the whole stack running in lieu of setting up services manually and then initializing a virtual environment. I much prefer taking the time to setup Docker Compose even for a simple project. I don't have to initialize a virtual environment, install packages, make sure the DB is running, sync with the DB, run the web server, etc. All of that work becomes automatic and I can focus on development instead sysadmin stuff.

ris 11y ago

Frankly I just use Nix, which is a little like having virtualenvs for your entire distribution and as such can handle non-python dependencies properly.

pekk 11y ago
Virtualenv isn't supposed to handle non-Python dependencies. Why use Nix instead of containers or VMs?
- ris 11y ago
  "Virtualenv isn't supposed to handle non-Python dependencies." Making it kinda useless for anything like numpy or scipy. "Why use Nix instead of containers or VMs?" Well, why use virtualenvs instead of containers or VMs? Because it's lighter weight and more straightforward. And doesn't require you to swap out your whole working environment every time you want a slightly different package setup. Containers and VMs are a bad excuse for "I don't have a hope of getting the dependencies of my setup under control so I'm just going to ball them all up into this great big binary image." Nix packages are incredibly flexible and composable in comparison.

jaxondu 11y ago

I don't need the extra in virtualenvwrapper, just use the build-in venv module in Python 3: $ python3 -m venv myvenv $ source myvenv/bin/activate