6 ms·

When I saw the headline I thought the article was going to be about the ridiculously confusing tooling landscape around python development. I'm never sure what

by lambic 5y ago

When I saw the headline I thought the article was going to be about the ridiculously confusing tooling landscape around python development. I'm never sure what package manager or virtual environment thingy I'm supposed to be using.

ed_elliott_asc 5y ago
I’ve settled on using private venv environments for each repo/app and that seems to work well enough tbh.
runjake 5y ago
> I'm never sure what package manager or virtual environment thingy I'm supposed to be using. pip3 and virtualenv. Both are pretty easy to learn. Everything else is pretty much an abstraction layer on top of one or both of the above.
- nomel 5y ago
  pip3 assumes you have python 3, and the correct version of it, already installed. virtualenv requires that you manually juggle command line contexts, by running some nested scripts. For virtualenv, you can set up bash script that detect when you cd into a folder, but wow.
orf 5y ago
Just use poetry[1], there isn’t any reason not to. You can create and manage virtualenvs by hand if you’d like, but why do that to yourself? 1. https://python-poetry.org/ https://python-poetry.org/
- fmakunbound 5y ago
  I sure want to believe this is the last tooling I’ll need to use for Python tooling nirvana, but that’s what they told me about pip, pip3, venv, vitrualenv, conda.. And one coworker just suggested I just run the script in Docker. Ugh
  - orf 5y ago
    I’m not sure anyone said pip or virtualenv where the end of the road for Python tooling nirvana. They are just building blocks, poetry (or other tools) stick them together. That being said, I’ve been using Python for 16 years now and for me poetry is pretty close to nirvana. Not perfect, but really good. Handles virtualenv management, locking, dependency specification, etc etc.
    - iamcreasy 5y ago
      What are the drawbacks of Poetry?
      - orf 5y ago
        If you’re not using PyPi (think a corporate Nexus install or a private repo) then locking can be slow. But that’s not necessarily Poetry’s fault, it’s that those private servers don’t expose the complete pypi JSON api. Uhh, I’m struggling to think of anything else. It makes requirements.txt based projects feel barbaric and that will make you feel bad working with them?
serial_dev 5y ago
Having three different approaches in the three replies to this comment is just perfect.
- burnished 5y ago
  It mirrors my own experience. Especially the answer that strives to be condescending, as if you are the only one who is having this problem.
- orf 5y ago
  It’s only two, and there are two different approaches you can take: 1. Use something that manages virtualenvs + packages for you. 2. Don’t use something that manages virtualenvs + packages for you. For 1 you have poetry or pipenv. For 2 you have… well virtualenv/pip. It’s really not that complex and I don’t get where this sentiment comes from.
  - bradrn 5y ago
    > It’s really not that complex and I don’t get where this sentiment comes from. You’ve already listed four separate applications: poetry, pipenv, virtualenv and pip. And if you look online you’ll also find Anaconda and probably several others. For a Python beginner, it’s confusing to try figure out the exact differences between them — let alone what exactly a ‘virtualenv’ is or why you would want to use it. By comparison, Haskell — a language with a reputation for poor dependency management — has only two competing package managers, and these are practically identical. Most other languages have only one ‘official’ package manager. No wonder people find Python dependency management confusing!
    - orf 5y ago
      So just to be clear, Python has one official package manager: pip. It’s built into Python. Both it and virtualenv are part of Python itself, they are not separately installed applications. It’s a bit like saying Rust is confusing because “rustc”, “cargo” and “clippy” are different things. So that’s the base level. You’ve got pip and virtualenv. Everything else builds ontop of that. But hey it turns out it would be cool to iterate on stuff outside of Python itself. And maybe there are some useful things we can add to pip and virtualenv? And so Poetry and Pipenv are born. It’s probably Stockholm syndrome but this seems pretty simple.
version_five 5y ago
> I'm never sure what package manager or virtual environment thingy I'm supposed to be using This feels a bit anti-intellectual to me. Like any other thing, it takes a bit of reading to figure our virtual environments, but its literally one line to create a new one one you know how (I almost always use Conda). I don't see how this is different than the setup used for any development, and I can't imagine it's a major impediment, certainly not a lasting one, to people working in python. You could always open a colab notebook if you really don't want to be bothered.
- lambic 5y ago
  > it takes a bit of reading to figure our virtual environments The problem is that depending on what article you read or what tutorial you follow you'll learn different things. I recently wanted to write something using flask. I followed 4 different tutorials and all 4 did things differently (and none of them mentioned Conda or Poetry). I had a similar experience writing a Telegram bot.
exdsq 5y ago
Just freeze requirements (pip freeze > requirements.txt) and load them into whatever env you're comfortable using. That's worked for me
- pid-1 5y ago
  My grip with pip freeze is that it doesn't differentiate dependencies from dependencies-of-dependencies. So you might end up in dependency hell when trying to remove or modify something.
kortex 5y ago
In short: on posix: pyenv, pipx, then virtualenv+pip or conda. Pyenv: tool to manage different python major/minor/patch versions. Pipx: installs CLI/GUI tools in their own virtualenv and links them to ~/.local/bin. Way more convenient if you have commonly used tools virtualenv: use to isolate python environments Pip: used to install packages Conda: basically does all the above, also handles linked libraries and a bunch of stuff. Annoying for deployment, but it's often the least worst option. Python-N.M: use pyenv to manage your python installations. Way cleaner than brew. Never use system python for things. Python2 thru 3.3: you really ought to be using py>3.6 by now, but it may not be your choice. I haven't in years but I think you can use pipx or "pip install --user virtualenv" to get the virtualenv binary. Then use virtualenv from there. Py>3.3: use "python -m venv" to manage virtualenvs. Use pipx to install cli tools. Do everything in venvs. On windows or if you rely on lots of academic packages: you are probably better off just using conda. Also the pre-commit tool manages its own virtualenv cache, which is cool for making repos easy to bootstrap. There's also poetry, but it's more new and if you are at the point in your python journey that you are asking these questions, then in my opinion it's likely to just muddy the waters more. And if anyone is curious "why the hell is python this way? I don't have this problem with $LANGUAGE?" It's because when python fires up, basically a small miracle occurs that is able to locate and execute .py, .pyc, .zip, several other file types, directories, compiled libraries, and basically anything you want if you override the module machinery.

by lambic 5y ago

ed_elliott_asc 5y ago

I’ve settled on using private venv environments for each repo/app and that seems to work well enough tbh.

runjake 5y ago

> I'm never sure what package manager or virtual environment thingy I'm supposed to be using. pip3 and virtualenv. Both are pretty easy to learn. Everything else is pretty much an abstraction layer on top of one or both of the above.

nomel 5y ago
pip3 assumes you have python 3, and the correct version of it, already installed. virtualenv requires that you manually juggle command line contexts, by running some nested scripts. For virtualenv, you can set up bash script that detect when you cd into a folder, but wow.

orf 5y ago

Just use poetry[1], there isn’t any reason not to. You can create and manage virtualenvs by hand if you’d like, but why do that to yourself? 1. https://python-poetry.org/ https://python-poetry.org/

fmakunbound 5y ago
I sure want to believe this is the last tooling I’ll need to use for Python tooling nirvana, but that’s what they told me about pip, pip3, venv, vitrualenv, conda.. And one coworker just suggested I just run the script in Docker. Ugh
- orf 5y ago
  I’m not sure anyone said pip or virtualenv where the end of the road for Python tooling nirvana. They are just building blocks, poetry (or other tools) stick them together. That being said, I’ve been using Python for 16 years now and for me poetry is pretty close to nirvana. Not perfect, but really good. Handles virtualenv management, locking, dependency specification, etc etc.
  - iamcreasy 5y ago
    What are the drawbacks of Poetry?
    - orf 5y ago
      If you’re not using PyPi (think a corporate Nexus install or a private repo) then locking can be slow. But that’s not necessarily Poetry’s fault, it’s that those private servers don’t expose the complete pypi JSON api. Uhh, I’m struggling to think of anything else. It makes requirements.txt based projects feel barbaric and that will make you feel bad working with them?

serial_dev 5y ago

Having three different approaches in the three replies to this comment is just perfect.

burnished 5y ago
It mirrors my own experience. Especially the answer that strives to be condescending, as if you are the only one who is having this problem.
orf 5y ago
It’s only two, and there are two different approaches you can take: 1. Use something that manages virtualenvs + packages for you. 2. Don’t use something that manages virtualenvs + packages for you. For 1 you have poetry or pipenv. For 2 you have… well virtualenv/pip. It’s really not that complex and I don’t get where this sentiment comes from.
- bradrn 5y ago
  > It’s really not that complex and I don’t get where this sentiment comes from. You’ve already listed four separate applications: poetry, pipenv, virtualenv and pip. And if you look online you’ll also find Anaconda and probably several others. For a Python beginner, it’s confusing to try figure out the exact differences between them — let alone what exactly a ‘virtualenv’ is or why you would want to use it. By comparison, Haskell — a language with a reputation for poor dependency management — has only two competing package managers, and these are practically identical. Most other languages have only one ‘official’ package manager. No wonder people find Python dependency management confusing!
  - orf 5y ago
    So just to be clear, Python has one official package manager: pip. It’s built into Python. Both it and virtualenv are part of Python itself, they are not separately installed applications. It’s a bit like saying Rust is confusing because “rustc”, “cargo” and “clippy” are different things. So that’s the base level. You’ve got pip and virtualenv. Everything else builds ontop of that. But hey it turns out it would be cool to iterate on stuff outside of Python itself. And maybe there are some useful things we can add to pip and virtualenv? And so Poetry and Pipenv are born. It’s probably Stockholm syndrome but this seems pretty simple.

version_five 5y ago

> I'm never sure what package manager or virtual environment thingy I'm supposed to be using This feels a bit anti-intellectual to me. Like any other thing, it takes a bit of reading to figure our virtual environments, but its literally one line to create a new one one you know how (I almost always use Conda). I don't see how this is different than the setup used for any development, and I can't imagine it's a major impediment, certainly not a lasting one, to people working in python. You could always open a colab notebook if you really don't want to be bothered.

lambic 5y ago
> it takes a bit of reading to figure our virtual environments The problem is that depending on what article you read or what tutorial you follow you'll learn different things. I recently wanted to write something using flask. I followed 4 different tutorials and all 4 did things differently (and none of them mentioned Conda or Poetry). I had a similar experience writing a Telegram bot.

exdsq 5y ago

Just freeze requirements (pip freeze > requirements.txt) and load them into whatever env you're comfortable using. That's worked for me

pid-1 5y ago
My grip with pip freeze is that it doesn't differentiate dependencies from dependencies-of-dependencies. So you might end up in dependency hell when trying to remove or modify something.

kortex 5y ago

In short: on posix: pyenv, pipx, then virtualenv+pip or conda. Pyenv: tool to manage different python major/minor/patch versions. Pipx: installs CLI/GUI tools in their own virtualenv and links them to ~/.local/bin. Way more convenient if you have commonly used tools virtualenv: use to isolate python environments Pip: used to install packages Conda: basically does all the above, also handles linked libraries and a bunch of stuff. Annoying for deployment, but it's often the least worst option. Python-N.M: use pyenv to manage your python installations. Way cleaner than brew. Never use system python for things. Python2 thru 3.3: you really ought to be using py>3.6 by now, but it may not be your choice. I haven't in years but I think you can use pipx or "pip install --user virtualenv" to get the virtualenv binary. Then use virtualenv from there. Py>3.3: use "python -m venv" to manage virtualenvs. Use pipx to install cli tools. Do everything in venvs. On windows or if you rely on lots of academic packages: you are probably better off just using conda. Also the pre-commit tool manages its own virtualenv cache, which is cool for making repos easy to bootstrap. There's also poetry, but it's more new and if you are at the point in your python journey that you are asking these questions, then in my opinion it's likely to just muddy the waters more. And if anyone is curious "why the hell is python this way? I don't have this problem with $LANGUAGE?" It's because when python fires up, basically a small miracle occurs that is able to locate and execute .py, .pyc, .zip, several other file types, directories, compiled libraries, and basically anything you want if you override the module machinery.