7 ms·

I genuinely don't understand how someone can be a "proponent" of pipenv. In the mangled space of python package maintenance, pipenv is probably the absolute wor

I genuinely don't understand how someone can be a "proponent" of pipenv. In the mangled space of python package maintenance, pipenv is probably the absolute worst solution - pipenv install updates your lockfile on install exactly like how `npm install` updates the package-lock.json and requires additional flags to use the lockfile instead - pipenv's dependency resolution is the slowest among all the tools in this space including poetry, pip-tools, hatch and so on - pipenv has consistently failed to do its job to simply install dependencies because of issues like the one described in the article where it fails to resolve changes in hash simply because a new wheel was added for the platform - Tons of longstanding bugs that fail to be resolved in any sort of timely manner like this one: https://github.com/pypa/pipenv/issues/2413 https://github.com/pypa/pipenv/issues/2413 I am sick and tired of the python packaging space pushing this godawful tool for Python. Please just use pip-tools or poetry instead.

323 4y ago
> exactly like how `npm install` updates the package-lock.json and requires additional flags to use the lockfile instead `npm install` doesn't do that anymore since at least a year
Galanwe 4y ago
Not to mention the horrible configuration anti-pattern where everything depends on environment variables. Environment variable tweaks are for local-only specific fixes, not for reproducible/distributable configuration.
- funklute 4y ago
  > not for reproducible/distributable configuration At my work, we use environment variables all the time for configurations, e.g. for telling a kubernetes pod where to find the database. In your opinion, what is the better option(s)?
  - xvector 4y ago
    Config files, and command line arguments as a very distant second
  - BerislavLopac 4y ago
    Secret managers. All cloud providers have them built in, and on other platforms you can easily whip up a self-hosted one. See https://blog.diogomonica.com//2017/03/27/why-you-shouldnt-use-env-variables-for-secret-data/ https://blog.diogomonica.com//2017/03/27/why-you-shouldnt-us...
    - xvector 4y ago
      I recommend against with using secrets managers as opposed to config files for storing configuration. Ideally, your configuration is versioned and code-reviewed. Config files are optimal. Even secrets used by your deployment should be stored in configuration - encrypted, of course. The benefit of versioned configuration files, secrets included, cannot be overstated. You can encrypt sensitive configuration fields with tools like `mozilla/sops` [1], which will reach out to your KMS or secrets manager of choice to encrypt/decrypt sensitive fields on the fly. This way, you are minimizing the splitting of state across secrets managers and your code. Your configuration is stored at code, code reviewed, and versioned, which has devops and security benefits of its own. [1]: https://github.com/mozilla/sops https://github.com/mozilla/sops
  - Galanwe 4y ago
    The problem with environment variables as configuration is that it's unstructured, hardly documented, and overall hard to reproduce and inspect. Nothing is worse than trying to understand an issue with a program that heavily relies on environment variables for configuration, as environment variables are designed to be short-lived, memory only. A good old configuration file is the best. You can version it, distribute it, it's explicit, possibly structured, easily documented. That doesn't mean that there is no room for environment variables, but these should be for local-only hacks and tweaks.
    - funklute 4y ago
- realharo 4y ago
  This is partially addressed by .env files (many projects support them) and an accompanying file with empty values or defaults, that lists and documents the variables (the .env file itself is then in .gitignore)
orra 4y ago
> I am sick and tired of the python packaging space pushing this godawful tool for Python. Please just use pip-tools or poetry instead. As an outsider, Python packaging seems like a shit show, sorry. This bug in pipenv is bad, yes. But poetry was deliberately non-deterministically failing CI builds that used a deprecated feature. Maybe pip-tools is reliable and has a reasonable maintainer, but having to manually create venvs is mediocre UX. PDM sounds interesting because it doesn't need venvs under the hood, thanks to a recent PEP.
- BerislavLopac 4y ago
  > Python packaging seems like a shit show It does, but primarily because most of the tools - and that includes pipenv, Poetry, pdm and hatch - have crammed up to three separate functionalities into one (probably following the example of npm): packaging on one hand, dependency management on the other, and virtual environments on the third. The packaging question has mostly been resolved: distribute your packages as wheels and install them using pip; that's all. Dependency managament inherently has more complexity, and adding virtual environments in the mix, especially with the introduction of PEP 582 (as you mention), doesn't help things.
  - coldtea 4y ago
    >primarily because most of the tools - and that includes pipenv, Poetry, pdm and hatch - have crammed up to three separate functionalities into one (probably following the example of npm): packaging on one hand, dependency management on the other, and virtual environments on the third. That never stopped npm to be way better than all of them... >The packaging question has mostly been resolved: distribute your packages as wheels and install them using pip; that's all. Dependency managament inherently has more complexity, and adding virtual environments in the mix, especially with the introduction of PEP 582 (as you mention), doesn't help things. So, the "solution" is "just give up"...
    - viraptor 4y ago
      Npm doesn't do environments. You have the choice of "in this directory" or "globally" and pray there are no symlinks involved or things fail mysteriously. It doesn't quite do packaging either. So it lands in a similar space to what the parent describes - mostly a good dependency manager.
avidphantasm 4y ago
Plus pipenv falls down the stairs whenever I have tried to use it with packages that rely on native extensions like NumPy and GDAL. Not sure about pip-tools or poetry. My path of least resistance has been to use a local package manager (MacPorts, Homebrew before that), and then use Conda in CICD to test compatibility with different versions of Python.
IceDane 4y ago
There is no good alternative. Maybe poetry is the least worst, but it is a fractal trash fire like the rest of the python ecosystem. To name some issues i've had with poetry(and keep having): - Updating packages does not properly take effect sometimes. It will sometimes keep using old code, and only deleting the entire underlying venv fixed it. In this instance, I was using a wheel package locally. - It has incredibly poor UX and will just vomit raw python errors when you use it slightly incorrectly. - There's so many bugs in its handling of CLI args and getting a typical python None error is very common - It is ridiculously slow and doesn't cache things properly I could probably go on but I'd have to sit and think a bit about it, and that's pretty much the opposite of what I want to do: forget all about python and poetry.
Tainnor 4y ago
> pipenv's dependency resolution is the slowest among all the tools in this space including poetry, pip-tools, hatch and so on "Slow" doesn't just mean "slightly annoying", mind you. I was once tasked to properly package up an Airflow installation. Not being a Python guy, I tried doing it with pipenv because we were using it for other projects. It was impossible to resolve the dependencies because the resolver would routinely throw a timeout error. In the end I gave up and switched to Poetry. That was still a bit slow (I think Airflow just has a messed up dependency graph...), but it wrote the lockfile in a reasonable amount of time and just worked thereafter...
gmmeyer 4y ago
Every solution in the space is bad and it's a wonder that any python software is maintainable and I think the only real answer is everyone gets fucked all the time with these bugs and rough edges I say this as someone who has done an unfortunately large amount of work in the python packaging and python package distribution space and still get bitten every few months
Doxin 4y ago
Personally i've never really seen the advantages any of these tools over having a requirements.txt, or slightly fancier and more modern, using a pyproject.toml file. e.g. in a recent project the entire package configuration is this pyproject.toml file: [project] name = "vera" version = "0.0.0" requires-python = ">=3.10" authors = [ { name = "Doxin" } ] dependencies = ["moderngl==5.7.3", "pysdl2==0.9.14", etc etc] [project.optional-dependencies] dev = ["pre-commit==2.20.0"] [tool.setuptools] packages = ["vera"] Getting started with development is as easy as python3.10 -m venv venv source venv/bin/activate pip install -e vera[dev] Updating packages is just rerunning the pip install command.

323 4y ago

> exactly like how `npm install` updates the package-lock.json and requires additional flags to use the lockfile instead `npm install` doesn't do that anymore since at least a year

Galanwe 4y ago

Not to mention the horrible configuration anti-pattern where everything depends on environment variables. Environment variable tweaks are for local-only specific fixes, not for reproducible/distributable configuration.

funklute 4y ago
> not for reproducible/distributable configuration At my work, we use environment variables all the time for configurations, e.g. for telling a kubernetes pod where to find the database. In your opinion, what is the better option(s)?
- xvector 4y ago
  Config files, and command line arguments as a very distant second
- BerislavLopac 4y ago
  Secret managers. All cloud providers have them built in, and on other platforms you can easily whip up a self-hosted one. See https://blog.diogomonica.com//2017/03/27/why-you-shouldnt-use-env-variables-for-secret-data/ https://blog.diogomonica.com//2017/03/27/why-you-shouldnt-us...
  - xvector 4y ago
    I recommend against with using secrets managers as opposed to config files for storing configuration. Ideally, your configuration is versioned and code-reviewed. Config files are optimal. Even secrets used by your deployment should be stored in configuration - encrypted, of course. The benefit of versioned configuration files, secrets included, cannot be overstated. You can encrypt sensitive configuration fields with tools like `mozilla/sops` [1], which will reach out to your KMS or secrets manager of choice to encrypt/decrypt sensitive fields on the fly. This way, you are minimizing the splitting of state across secrets managers and your code. Your configuration is stored at code, code reviewed, and versioned, which has devops and security benefits of its own. [1]: https://github.com/mozilla/sops https://github.com/mozilla/sops
- Galanwe 4y ago
  The problem with environment variables as configuration is that it's unstructured, hardly documented, and overall hard to reproduce and inspect. Nothing is worse than trying to understand an issue with a program that heavily relies on environment variables for configuration, as environment variables are designed to be short-lived, memory only. A good old configuration file is the best. You can version it, distribute it, it's explicit, possibly structured, easily documented. That doesn't mean that there is no room for environment variables, but these should be for local-only hacks and tweaks.
  - funklute 4y ago
realharo 4y ago
This is partially addressed by .env files (many projects support them) and an accompanying file with empty values or defaults, that lists and documents the variables (the .env file itself is then in .gitignore)

orra 4y ago

> I am sick and tired of the python packaging space pushing this godawful tool for Python. Please just use pip-tools or poetry instead. As an outsider, Python packaging seems like a shit show, sorry. This bug in pipenv is bad, yes. But poetry was deliberately non-deterministically failing CI builds that used a deprecated feature. Maybe pip-tools is reliable and has a reasonable maintainer, but having to manually create venvs is mediocre UX. PDM sounds interesting because it doesn't need venvs under the hood, thanks to a recent PEP.

BerislavLopac 4y ago
> Python packaging seems like a shit show It does, but primarily because most of the tools - and that includes pipenv, Poetry, pdm and hatch - have crammed up to three separate functionalities into one (probably following the example of npm): packaging on one hand, dependency management on the other, and virtual environments on the third. The packaging question has mostly been resolved: distribute your packages as wheels and install them using pip; that's all. Dependency managament inherently has more complexity, and adding virtual environments in the mix, especially with the introduction of PEP 582 (as you mention), doesn't help things.
- coldtea 4y ago
  >primarily because most of the tools - and that includes pipenv, Poetry, pdm and hatch - have crammed up to three separate functionalities into one (probably following the example of npm): packaging on one hand, dependency management on the other, and virtual environments on the third. That never stopped npm to be way better than all of them... >The packaging question has mostly been resolved: distribute your packages as wheels and install them using pip; that's all. Dependency managament inherently has more complexity, and adding virtual environments in the mix, especially with the introduction of PEP 582 (as you mention), doesn't help things. So, the "solution" is "just give up"...
  - viraptor 4y ago
    Npm doesn't do environments. You have the choice of "in this directory" or "globally" and pray there are no symlinks involved or things fail mysteriously. It doesn't quite do packaging either. So it lands in a similar space to what the parent describes - mostly a good dependency manager.

avidphantasm 4y ago

Plus pipenv falls down the stairs whenever I have tried to use it with packages that rely on native extensions like NumPy and GDAL. Not sure about pip-tools or poetry. My path of least resistance has been to use a local package manager (MacPorts, Homebrew before that), and then use Conda in CICD to test compatibility with different versions of Python.

IceDane 4y ago

There is no good alternative. Maybe poetry is the least worst, but it is a fractal trash fire like the rest of the python ecosystem. To name some issues i've had with poetry(and keep having): - Updating packages does not properly take effect sometimes. It will sometimes keep using old code, and only deleting the entire underlying venv fixed it. In this instance, I was using a wheel package locally. - It has incredibly poor UX and will just vomit raw python errors when you use it slightly incorrectly. - There's so many bugs in its handling of CLI args and getting a typical python None error is very common - It is ridiculously slow and doesn't cache things properly I could probably go on but I'd have to sit and think a bit about it, and that's pretty much the opposite of what I want to do: forget all about python and poetry.

Tainnor 4y ago

> pipenv's dependency resolution is the slowest among all the tools in this space including poetry, pip-tools, hatch and so on "Slow" doesn't just mean "slightly annoying", mind you. I was once tasked to properly package up an Airflow installation. Not being a Python guy, I tried doing it with pipenv because we were using it for other projects. It was impossible to resolve the dependencies because the resolver would routinely throw a timeout error. In the end I gave up and switched to Poetry. That was still a bit slow (I think Airflow just has a messed up dependency graph...), but it wrote the lockfile in a reasonable amount of time and just worked thereafter...

gmmeyer 4y ago

Every solution in the space is bad and it's a wonder that any python software is maintainable and I think the only real answer is everyone gets fucked all the time with these bugs and rough edges I say this as someone who has done an unfortunately large amount of work in the python packaging and python package distribution space and still get bitten every few months

Doxin 4y ago

Personally i've never really seen the advantages any of these tools over having a requirements.txt, or slightly fancier and more modern, using a pyproject.toml file. e.g. in a recent project the entire package configuration is this pyproject.toml file: [project] name = "vera" version = "0.0.0" requires-python = ">=3.10" authors = [ { name = "Doxin" } ] dependencies = ["moderngl==5.7.3", "pysdl2==0.9.14", etc etc] [project.optional-dependencies] dev = ["pre-commit==2.20.0"] [tool.setuptools] packages = ["vera"] Getting started with development is as easy as python3.10 -m venv venv source venv/bin/activate pip install -e vera[dev] Updating packages is just rerunning the pip install command.