5 ms·

The big difference is that npm will automatically (since 2017) save a version range to the project metadata, and will automatically create this metadata file if

by Timon3 2y ago

The big difference is that npm will automatically (since 2017) save a version range to the project metadata, and will automatically create this metadata file if it doesn't exist. Same for other package managers in the Node world. I just installed Python 3.13 with pip 24.2, created a venv and installed a package - and nothing, no file was created and nothing was saved. Even if I touch requirements.txt and pyproject.toml, pip doesn't save anything about the package. This creates a massive gap in usability of projects by people not very familiar with the languages. Node-based projects sometimes have issues because dependencies changed without respecting semver, but Python projects often can't be installed and you have no idea why without spending lots of time looking through versions. Of course there are other package managers for Python that do this better, but pip is still the de-facto default and is often used in tutorials for new developers. Hopefully uv can improve things!

elashri 2y ago
> Of course there are other package managers for Python that do this better I think if you are comparing with what NPM does then you would have to say that native pip can do that too. It is just one command `pip freeze > requirements.txt` It does include everything in the venv (or in you environment in general) but if you stick to only add required things (one venv for each project) then you will get requirements.txt files
- whyever 2y ago
  I don't think this is the same. Does it also cover transitive dependencies?
  - elashri 2y ago
    Sorry if what I said about NPM is not accurate. But in reality if you are pinning the dependencies (all of them actual get pinned) then when pip is installing it will grab the correct version of the transitive dependency (both packages are pinned) So I am not sure when this will become a problem.
- Timon3 2y ago
  Sure, you can manually do that. But my point is that pip doesn't do this automatically, and that is what makes so many Python projects essentially unusable without investing massive amounts of time into debugging. Good defaults and good standards matter a lot.
  - elashri 2y ago
    > without investing massive amounts of time into debugging Again even if you are going to spend sometime to learn something that will have better tool for doing that like uv and poetry package managers. This is no massive amount of time. And eveb pip freeze is just one standard command and will give you portable environment be everything will be pinned in your environment. You just don't want to do everything with system global environment which is a common sense not a huge ask. So I am not sure what is the massive amount of debugging needes for that.
    - Timon3 2y ago
      I am specifically talking about the scenario where I, someone experienced with Python, am trying to use a project from someone who is not experienced in Python. There is nothing I can change since the project was already created some time ago. Most of the time, the only way for me to get the project running is to spend a lot of time debugging version issues - to find combinations of dependencies that are installable, and then to see if they work in the application context, and then to see if any bugs are part of the application or the dependency. You might explain that away by asking why I'd want to run these projects, but a large percentage of data science projects I've tried to run end up in this scenario, and it simply doesn't happen with npm. A command existing is not a valid replacement for good defaults, since it literally affects my ability to run these projects.
__mharrison__ 2y ago
I recommend to start using UV. It is very fast and tracks the libraries you are using. After years of venv/pip, I'm not going back (unless a client requires it).
- jessekv 2y ago
  Another nice thing about uv is it can install python itself in the venv. So no need to mess around with brew/deadsnakes and multiple global python versions on your dev system. This is actually an improvement over the node/nvm approach.
VagabundoP 2y ago
All that can be specified in a pyproject.toml. As some posters mentioned uv takes care of a lot of that and you can even pin it to a version of python. If it’s just a one off script you can get all the dep information in the script header and uv can take care of all the venv/deps for you if you transfer the script to another machine by reading the headers in a comment section at the start of the script. All this is based on PEPs to standardise packaging. It’s slow but moving in the right direction.
- Timon3 2y ago
  What do I have to put into pyproject.toml so that pip saves dependency ranges by default?
  - VagabundoP 2y ago
    So pyproject.toml will be used by uv and others, like poetry. Pip uses a requirements.txt for depandancy management. Using uv as an example[0]: uv add "tqdm >=4.66.2,<5" [0]https://docs.astral.sh/uv/concepts/dependencies/#project-dependencies https://docs.astral.sh/uv/concepts/dependencies/#project-dep... I don't have access to uv to test that command at the moment, but that should work. uv then installs the dependency in the .venv directory in the project directory. This may include a specific version of python as well, if you pin one.
    - Timon3 2y ago
      As I already replied to another user, I know that this exists, and it doesn't have anything to do with my point. So sadly your suggestion doesn't help in any way.

by Timon3 2y ago

elashri 2y ago

> Of course there are other package managers for Python that do this better I think if you are comparing with what NPM does then you would have to say that native pip can do that too. It is just one command `pip freeze > requirements.txt` It does include everything in the venv (or in you environment in general) but if you stick to only add required things (one venv for each project) then you will get requirements.txt files

whyever 2y ago
I don't think this is the same. Does it also cover transitive dependencies?
- elashri 2y ago
  Sorry if what I said about NPM is not accurate. But in reality if you are pinning the dependencies (all of them actual get pinned) then when pip is installing it will grab the correct version of the transitive dependency (both packages are pinned) So I am not sure when this will become a problem.
Timon3 2y ago
Sure, you can manually do that. But my point is that pip doesn't do this automatically, and that is what makes so many Python projects essentially unusable without investing massive amounts of time into debugging. Good defaults and good standards matter a lot.
- elashri 2y ago
  > without investing massive amounts of time into debugging Again even if you are going to spend sometime to learn something that will have better tool for doing that like uv and poetry package managers. This is no massive amount of time. And eveb pip freeze is just one standard command and will give you portable environment be everything will be pinned in your environment. You just don't want to do everything with system global environment which is a common sense not a huge ask. So I am not sure what is the massive amount of debugging needes for that.
  - Timon3 2y ago
    I am specifically talking about the scenario where I, someone experienced with Python, am trying to use a project from someone who is not experienced in Python. There is nothing I can change since the project was already created some time ago. Most of the time, the only way for me to get the project running is to spend a lot of time debugging version issues - to find combinations of dependencies that are installable, and then to see if they work in the application context, and then to see if any bugs are part of the application or the dependency. You might explain that away by asking why I'd want to run these projects, but a large percentage of data science projects I've tried to run end up in this scenario, and it simply doesn't happen with npm. A command existing is not a valid replacement for good defaults, since it literally affects my ability to run these projects.

__mharrison__ 2y ago

I recommend to start using UV. It is very fast and tracks the libraries you are using. After years of venv/pip, I'm not going back (unless a client requires it).

jessekv 2y ago
Another nice thing about uv is it can install python itself in the venv. So no need to mess around with brew/deadsnakes and multiple global python versions on your dev system. This is actually an improvement over the node/nvm approach.

VagabundoP 2y ago

All that can be specified in a pyproject.toml. As some posters mentioned uv takes care of a lot of that and you can even pin it to a version of python. If it’s just a one off script you can get all the dep information in the script header and uv can take care of all the venv/deps for you if you transfer the script to another machine by reading the headers in a comment section at the start of the script. All this is based on PEPs to standardise packaging. It’s slow but moving in the right direction.

Timon3 2y ago
What do I have to put into pyproject.toml so that pip saves dependency ranges by default?
- VagabundoP 2y ago
  So pyproject.toml will be used by uv and others, like poetry. Pip uses a requirements.txt for depandancy management. Using uv as an example[0]: uv add "tqdm >=4.66.2,<5" [0]https://docs.astral.sh/uv/concepts/dependencies/#project-dependencies https://docs.astral.sh/uv/concepts/dependencies/#project-dep... I don't have access to uv to test that command at the moment, but that should work. uv then installs the dependency in the .venv directory in the project directory. This may include a specific version of python as well, if you pin one.
  - Timon3 2y ago
    As I already replied to another user, I know that this exists, and it doesn't have anything to do with my point. So sadly your suggestion doesn't help in any way.