6 ms·

This is a reminder that the excellent pyenv [0] project can help you manage all your Python versions. - Set global and per-project python versions - Not writt

by tbran 4y ago

This is a reminder that the excellent pyenv [0] project can help you manage all your Python versions. - Set global and per-project python versions - Not written in Python. - Shims your PATH - Linux, Mac, Windows [0]: https://github.com/pyenv/pyenv https://github.com/pyenv/pyenv

Already__Taken 4y ago
Not really windows, WSL. There's windows fork recommended. Classic python splitting the ecosystem up. Why something like this isn't in go or rust that's actually cross platform capable just seems like excess effort.
- Spivak 4y ago
  Because thanks to WSL the lingua franca of scripting languages is bash and that’s how pyenv started. Rewriting when it works fine on Windows and every dev I know uses WSL anyway is excess effort. Hell, the fork could have used Go or whatever too but it went with a bunch of .bat scripts.
  - 0cf8612b2e1e 4y ago
    Hello from corporate America where WSL is most certainly not standard and will require a virgin sacrifice if you want to get it approved by IT. Which is to say, no pyenv is not an option for everyone.
    - ihaveajob 4y ago
      Indeed, I recently worked on a fairly simple deployment for a state government agency. When I described the process to their IT staff and mentioned that we prefer using WSL, they looked at me like I had grown 2 heads.
    - int_19h 4y ago
      Even aside from that, WSL is only useful if you are writing code that will run on Linux at the end of the day. This is usually true for web apps, but that's not the only thing people write in Python. Libraries, in particular, need to be able to target Windows directly.
      - Already__Taken 4y ago
        or even if you want to build a python binary for windows, you can't cross compile it as far as I've found
gvalkov 4y ago
A hidden gem of pyenv is its 'python-build' plugin, which just lets you build and install any Python version in the least number of steps: git clone https://github.com/pyenv/pyenv.git cd pyenv/plugins/python-build/bin ./python-build --definitions ./python-build 3.10.8 /opt/python/3.10.8 PYTHON_CONFIGURE_OPTS="--enable-shared" ./python-build 3.10.8 /opt/python/3.10.8
- techdragon 4y ago
  I’ve regularly used ‘python-build’ in docker container construction to build a minimal (or customised) self contained Python binary, libs tool chain, etc, in an isolated path to make copying to from the build container to the final artefact container. It’s just all around excellent tooling.
ImageXav 4y ago
How does pyenv compare with conda, mamba, pip and poetry? I typically use conda but ever since they broke with python 3.10 I have been considering moving to other environment managers.
- persedes 4y ago
  Try Pyenv for the Python version and poetry for the venv and dependency management. Once you're used to the workflow it's pretty smooth. (We switched from conda 2 years ago)
- Noumenon72 4y ago
  According to StackOverflow the conda 3.10 bug was fixed this January. https://stackoverflow.com/a/70614013/733092 https://stackoverflow.com/a/70614013/733092
- whimsicalism 4y ago
  I have not been able to understand why conda is so popular. I have no trouble with scientific computing with plain old `pyenv` and have never had any issues with C extensions or compilation with plain old `pip`. Conda seems to be the most prone to getting in weird states or just hanging while "Solving environment." I have been happier leaving it behind. Really the only two I would even consider using now are pyenv and poetry.
  - int_19h 4y ago
    What's your platform? Issues with compilation are far more common on Windows, for example. The other thing about Conda is that it doesn't cover just Python and Python packages, but all kinds of software that may be directly or indirectly relevant. For example, on Windows, there are Conda packages for the Windows SDK, and for various C++ compilers.
vorticalbox 4y ago
as of late I've settled on using asdf[0] rather than nvm, pyvenv etc. [0] https://asdf-vm.com/guide/getting-started.html https://asdf-vm.com/guide/getting-started.html
- adammarples 4y ago
  Agreed, asdf is the last stop for me, and uses pyenv internally anyway for python
StreamBright 4y ago
Is there a problem with venv? I am seriously curious. I use venv for 10 years and so far it was able to do everything I ever wanted.
- okasaki 4y ago
  There's no problem with venv. Python is just a popular language and people have built a lot of tools.
- Kwpolska 4y ago
  venv is great, but it requires the Python version you want already installed. This is pyenv's job.
wodenokoto 4y ago
> Set global and per-project python versions Does that mean I can do things like install black and jupyter once and use that install across projects?
- whimsicalism 4y ago
  Yes? I mean that's just how standard python works as well.
  - wodenokoto 4y ago
    I use anaconda and every time I create a new environment I have to install jupyter and black in order to have access to them after running `conda activate $envname`.
zzleeper 4y ago
> Pyenv does not officially support Windows and does not work in Windows outside the Windows Subsystem for Linux. I don't only use Win but I do use windows, so having to use different tooling makes pyenv a hard to swallow pill
- BerislavLopac 4y ago
  There is a fork: https://github.com/pyenv-win/pyenv-win https://github.com/pyenv-win/pyenv-win
- imiller 4y ago
  There is a windows port that works great! https://pyenv-win.github.io/pyenv-win/ https://pyenv-win.github.io/pyenv-win/
synergy20 4y ago
`python3.10 -m venv` or `python3.11 -m venv` the default approach works fine and I don't need pip anything to get a virtualenv, what's the selling point of pyenv?
- sciurus 4y ago
  In your example, how did you install python 3.10 and 3.11? That's part of what pyenv solves for you.
  - Rufbdbskrufb473 4y ago
    Dont't most linux distros have separate packages for most recent python major versions, so that they can't installed in parallel?
    - lvass 4y ago
      Nix has every maintained python version, so does Arch User Repository, and those can definitely be in parallel.
    - int_19h 4y ago
      For most major distros, the oldest version of Python that they ship is usually the one that's used by other packages that depend on Python. It's not uncommon - especially for libraries - to need testing on something older.
  - synergy20 4y ago
    it's very easy to install them, probably easier than learning pyenv but I have not tried. I just want to use the default settings as much as possible, they're guaranteed to stay as long as python is alive and typically have less surprises for me on daily coding.
    - Kwpolska 4y ago
      Whether it's easy or not depends on the OS you're working on. On Windows, it's just another executable installer, so it's trivial. On macOS, the official installers are terrible (they have no uninstaller), so you want another way. On Linux etc., some distros have only one Python version in the repos, some have two or three, but the latest versions of all reasonable distros won't let you install Python 3.4.
fred123 4y ago
Or just use Conda/Mamba.
shadycuz 4y ago
I love pyenv but hate that it forces me to specify the patch version when installing python. Most of the time I wish I could just enter 3.9 and have it give me the latest.
ThreePinkApples 4y ago
I should maybe test out pyenv again. I tried it several years ago but I ran into some limitations. Or at least I thought I did, I might just have used it wrong. My main usecase, back then at least, was to have just one global-like environment for every Python version I was working with. And sometimes and additional one for debugging/testing some different packagecombinations. Back then it seemed like, to me, that pyenv only worked on a per-project basis, but I might just have followed incomplete guides and not looked into it deep enough.

by tbran 4y ago

Already__Taken 4y ago

Not really windows, WSL. There's windows fork recommended. Classic python splitting the ecosystem up. Why something like this isn't in go or rust that's actually cross platform capable just seems like excess effort.

Spivak 4y ago
Because thanks to WSL the lingua franca of scripting languages is bash and that’s how pyenv started. Rewriting when it works fine on Windows and every dev I know uses WSL anyway is excess effort. Hell, the fork could have used Go or whatever too but it went with a bunch of .bat scripts.
- 0cf8612b2e1e 4y ago
  Hello from corporate America where WSL is most certainly not standard and will require a virgin sacrifice if you want to get it approved by IT. Which is to say, no pyenv is not an option for everyone.
  - ihaveajob 4y ago
    Indeed, I recently worked on a fairly simple deployment for a state government agency. When I described the process to their IT staff and mentioned that we prefer using WSL, they looked at me like I had grown 2 heads.
  - int_19h 4y ago
    Even aside from that, WSL is only useful if you are writing code that will run on Linux at the end of the day. This is usually true for web apps, but that's not the only thing people write in Python. Libraries, in particular, need to be able to target Windows directly.
    - Already__Taken 4y ago
      or even if you want to build a python binary for windows, you can't cross compile it as far as I've found

gvalkov 4y ago

A hidden gem of pyenv is its 'python-build' plugin, which just lets you build and install any Python version in the least number of steps: git clone https://github.com/pyenv/pyenv.git cd pyenv/plugins/python-build/bin ./python-build --definitions ./python-build 3.10.8 /opt/python/3.10.8 PYTHON_CONFIGURE_OPTS="--enable-shared" ./python-build 3.10.8 /opt/python/3.10.8

techdragon 4y ago
I’ve regularly used ‘python-build’ in docker container construction to build a minimal (or customised) self contained Python binary, libs tool chain, etc, in an isolated path to make copying to from the build container to the final artefact container. It’s just all around excellent tooling.

ImageXav 4y ago

How does pyenv compare with conda, mamba, pip and poetry? I typically use conda but ever since they broke with python 3.10 I have been considering moving to other environment managers.

persedes 4y ago
Try Pyenv for the Python version and poetry for the venv and dependency management. Once you're used to the workflow it's pretty smooth. (We switched from conda 2 years ago)
Noumenon72 4y ago
According to StackOverflow the conda 3.10 bug was fixed this January. https://stackoverflow.com/a/70614013/733092 https://stackoverflow.com/a/70614013/733092
whimsicalism 4y ago
I have not been able to understand why conda is so popular. I have no trouble with scientific computing with plain old `pyenv` and have never had any issues with C extensions or compilation with plain old `pip`. Conda seems to be the most prone to getting in weird states or just hanging while "Solving environment." I have been happier leaving it behind. Really the only two I would even consider using now are pyenv and poetry.
- int_19h 4y ago
  What's your platform? Issues with compilation are far more common on Windows, for example. The other thing about Conda is that it doesn't cover just Python and Python packages, but all kinds of software that may be directly or indirectly relevant. For example, on Windows, there are Conda packages for the Windows SDK, and for various C++ compilers.

vorticalbox 4y ago

as of late I've settled on using asdf[0] rather than nvm, pyvenv etc. [0] https://asdf-vm.com/guide/getting-started.html https://asdf-vm.com/guide/getting-started.html

adammarples 4y ago
Agreed, asdf is the last stop for me, and uses pyenv internally anyway for python

StreamBright 4y ago

Is there a problem with venv? I am seriously curious. I use venv for 10 years and so far it was able to do everything I ever wanted.

okasaki 4y ago
There's no problem with venv. Python is just a popular language and people have built a lot of tools.
Kwpolska 4y ago
venv is great, but it requires the Python version you want already installed. This is pyenv's job.

wodenokoto 4y ago

> Set global and per-project python versions Does that mean I can do things like install black and jupyter once and use that install across projects?

whimsicalism 4y ago
Yes? I mean that's just how standard python works as well.
- wodenokoto 4y ago
  I use anaconda and every time I create a new environment I have to install jupyter and black in order to have access to them after running `conda activate $envname`.

zzleeper 4y ago

> Pyenv does not officially support Windows and does not work in Windows outside the Windows Subsystem for Linux. I don't only use Win but I do use windows, so having to use different tooling makes pyenv a hard to swallow pill

BerislavLopac 4y ago
There is a fork: https://github.com/pyenv-win/pyenv-win https://github.com/pyenv-win/pyenv-win
imiller 4y ago
There is a windows port that works great! https://pyenv-win.github.io/pyenv-win/ https://pyenv-win.github.io/pyenv-win/

synergy20 4y ago

`python3.10 -m venv` or `python3.11 -m venv` the default approach works fine and I don't need pip anything to get a virtualenv, what's the selling point of pyenv?

sciurus 4y ago
In your example, how did you install python 3.10 and 3.11? That's part of what pyenv solves for you.
- Rufbdbskrufb473 4y ago
  Dont't most linux distros have separate packages for most recent python major versions, so that they can't installed in parallel?
  - lvass 4y ago
    Nix has every maintained python version, so does Arch User Repository, and those can definitely be in parallel.
  - int_19h 4y ago
    For most major distros, the oldest version of Python that they ship is usually the one that's used by other packages that depend on Python. It's not uncommon - especially for libraries - to need testing on something older.
- synergy20 4y ago
  it's very easy to install them, probably easier than learning pyenv but I have not tried. I just want to use the default settings as much as possible, they're guaranteed to stay as long as python is alive and typically have less surprises for me on daily coding.
  - Kwpolska 4y ago
    Whether it's easy or not depends on the OS you're working on. On Windows, it's just another executable installer, so it's trivial. On macOS, the official installers are terrible (they have no uninstaller), so you want another way. On Linux etc., some distros have only one Python version in the repos, some have two or three, but the latest versions of all reasonable distros won't let you install Python 3.4.

fred123 4y ago

Or just use Conda/Mamba.

shadycuz 4y ago

I love pyenv but hate that it forces me to specify the patch version when installing python. Most of the time I wish I could just enter 3.9 and have it give me the latest.

ThreePinkApples 4y ago

I should maybe test out pyenv again. I tried it several years ago but I ran into some limitations. Or at least I thought I did, I might just have used it wrong. My main usecase, back then at least, was to have just one global-like environment for every Python version I was working with. And sometimes and additional one for debugging/testing some different packagecombinations. Back then it seemed like, to me, that pyenv only worked on a per-project basis, but I might just have followed incomplete guides and not looked into it deep enough.