6 ms·

What is the best way for an author of a Python package to develop and test on multiple versions of Python (e.g. 3.6, 3.7, 3.8, 3.9, 3.10), and be able to switch

What is the best way for an author of a Python package to develop and test on multiple versions of Python (e.g. 3.6, 3.7, 3.8, 3.9, 3.10), and be able to switch easily among the various Python versions? Every time I try to research this, I get lost in the chaos of Python packaging and environments. Currently I do a `pip3 install -e .`, which uses the default Python provided by the OS. Then I hope that my continuous integration matrix catches any problems in the other Python versions.

StopHammoTime 4y ago
asdf is good, and supports more than Python https://asdf-vm.com/ https://asdf-vm.com/
maest 4y ago
I think tox was designed to solve that problem. https://pypi.org/project/tox/ https://pypi.org/project/tox/
- Izkata 4y ago
  This, absolutely. If you're running tests for building a package, here's an entire "tox.ini" file for running all your tests on multiple python versions: [tox] envlist = py27,py36,py38 [testenv] deps = pytest # Install extra packages for testing separate from what's in setup.py or whatever commands = pytest # Whatever test command your project uses That's really it for bare minimum. If you have pytest configuration, they can also go into tox.ini under a [pytest] section to keep them in the same place. Instead of installing tox globally you can even install it to a separate virtualenv, activate it, then run "tox". It figures out the right thing to do. If you're not building a package: [tox] envlist = py27,py36,py38 skipsdist = True # Don't attempt to build a package / pull from setup.py [testenv] deps = -r requirements.txt # Pull in project requirements pytest commands = pytest
nicoburns 4y ago
Not an expert in python, but there is pyenv. I personally use asdf, which is great because it manages versions for multiple language runtimes.
ellisv 4y ago
use pyenv (https://github.com/pyenv/pyenv https://github.com/pyenv/pyenv) to manage multiple python versions (e.g. install 3.6, 3.7, etc) and then use virtual environments or use containers and a docker-compose file to run your test suite across versions
jnwatson 4y ago
pyenv is my tool of choice here. It allows one to easily install multiple versions of Python from source and switch among them. It has mostly avoided feature creep, which is a big plus among all the competing solutions.
shadycuz 4y ago
You need `pyenv` and `nox`. Just read this entire guide https://cjolowicz.github.io/posts/hypermodern-python-01-setup/ https://cjolowicz.github.io/posts/hypermodern-python-01-setu...
nl 4y ago
If you scroll down in the linked article it says: > ALWAYS use an environment! Don't install into your global interpreter! ... > When you need to create an environment for a project I personally always reach for venv and virtual environments. It's included in Python's stdlib so it's always available via python -m venv (as long as you are not on Debian/Ubuntu, otherwise you may have to install the python3-venv apt package) I think this is good advice, and it's pretty easy to do, and you can have named environments (via directory names) for each version you can switch between easily. Very occasionally some python libraries will use a globally installed system library or a particular version which changes per python version (I think maybe OpenCV has this problem on Windows or something?) and so then you have to reach for containers.
- martopix 4y ago
  Environments don't help you if you want to use different versions of Python.
bxparks 4y ago
Just want to thank everyone who responded with suggestions. I think it's amusing that there seems to be at least 5 different solutions to this problem: pyenv, nox, tox, asdf, docker.
imgabe 4y ago
A lot of people will recommend pyenv, but I find this ends up causing obscure problems as it messes with your PYTHONPATH. I like to just install alternative versions from source. Download the tarball you want from python.org and run ./configure make make altinstall and it will be installed separately without overwriting any other version. Then you can just create a venv to use with that version like python3.6 -m venv py36

StopHammoTime 4y ago

asdf is good, and supports more than Python https://asdf-vm.com/ https://asdf-vm.com/

maest 4y ago

I think tox was designed to solve that problem. https://pypi.org/project/tox/ https://pypi.org/project/tox/

Izkata 4y ago
This, absolutely. If you're running tests for building a package, here's an entire "tox.ini" file for running all your tests on multiple python versions: [tox] envlist = py27,py36,py38 [testenv] deps = pytest # Install extra packages for testing separate from what's in setup.py or whatever commands = pytest # Whatever test command your project uses That's really it for bare minimum. If you have pytest configuration, they can also go into tox.ini under a [pytest] section to keep them in the same place. Instead of installing tox globally you can even install it to a separate virtualenv, activate it, then run "tox". It figures out the right thing to do. If you're not building a package: [tox] envlist = py27,py36,py38 skipsdist = True # Don't attempt to build a package / pull from setup.py [testenv] deps = -r requirements.txt # Pull in project requirements pytest commands = pytest

nicoburns 4y ago

Not an expert in python, but there is pyenv. I personally use asdf, which is great because it manages versions for multiple language runtimes.

ellisv 4y ago

use pyenv (https://github.com/pyenv/pyenv https://github.com/pyenv/pyenv) to manage multiple python versions (e.g. install 3.6, 3.7, etc) and then use virtual environments or use containers and a docker-compose file to run your test suite across versions

jnwatson 4y ago

pyenv is my tool of choice here. It allows one to easily install multiple versions of Python from source and switch among them. It has mostly avoided feature creep, which is a big plus among all the competing solutions.

shadycuz 4y ago

You need `pyenv` and `nox`. Just read this entire guide https://cjolowicz.github.io/posts/hypermodern-python-01-setup/ https://cjolowicz.github.io/posts/hypermodern-python-01-setu...

nl 4y ago

If you scroll down in the linked article it says: > ALWAYS use an environment! Don't install into your global interpreter! ... > When you need to create an environment for a project I personally always reach for venv and virtual environments. It's included in Python's stdlib so it's always available via python -m venv (as long as you are not on Debian/Ubuntu, otherwise you may have to install the python3-venv apt package) I think this is good advice, and it's pretty easy to do, and you can have named environments (via directory names) for each version you can switch between easily. Very occasionally some python libraries will use a globally installed system library or a particular version which changes per python version (I think maybe OpenCV has this problem on Windows or something?) and so then you have to reach for containers.

martopix 4y ago
Environments don't help you if you want to use different versions of Python.

bxparks 4y ago

Just want to thank everyone who responded with suggestions. I think it's amusing that there seems to be at least 5 different solutions to this problem: pyenv, nox, tox, asdf, docker.

imgabe 4y ago

A lot of people will recommend pyenv, but I find this ends up causing obscure problems as it messes with your PYTHONPATH. I like to just install alternative versions from source. Download the tarball you want from python.org and run ./configure make make altinstall and it will be installed separately without overwriting any other version. Then you can just create a venv to use with that version like python3.6 -m venv py36