6 ms·

> Times have changed. The introduction of the cheese shop (PyPI), setuptools, and later pip, it became simple and straight forward to download and install packa

by jl6 7y ago

> Times have changed. The introduction of the cheese shop (PyPI), setuptools, and later pip, it became simple and straight forward to download and install packages. I’ve been out of the Python loop for a few years, but my last impression was that packaging and distribution of Python modules was far from a solved problem. Has this changed?

carlmr 7y ago
No, it's still one of the worst package management systems out there. I'm always wondering if it has to do with the language and or just the package manager itself. Rust's cargo, Node's npm and probably quite a few others exist that work amazingly well.
- alexchamberlain 7y ago
  What makes it the worst? Is it non-pure Python extensions, or is there something missing for pure python packages too?
  - lmm 7y ago
    It's broken for pure python too. The standards were too heavily influenced by traditional unix sysadmins who love server-global libraries, so that's still the direction all the tooling guides you in despite being the worst way to work with Python. Every couple of years they rewrite it without fixing any of the problems. Sometimes they make it worse, as with e.g. moving pip into python. I struggle to imagine the decision-making process that could come up with these outcomes - surely they must be studying what works for other languages or there'd be no way they'd manage to avoid it all so perfectly.
    - sametmax 7y ago
      Not anymore, .whl files are now similar to npm packages, with static metadata and just unpacked on pip installed. Most popular packages are .whl now.
- bow_ 7y ago
  Why do you think it is the worst? If you don't consider Python, what would you then think is the worst? Genuinely curious. I use Python daily and I rarely encounter problems with it. There is some getting used to in the beginning, but I went through a similar phase when I started using npm and cargo as well. What I can say is that I had to go through a lot of experimentation myself to arrive at the tools I use now (pyenv + Poetry). And if anything, maybe the lack of one way that is adopted by everyone in the community is the problem.
  - dagw 7y ago
    What I can say is that I had to go through a lot of experimentation myself to arrive at the tools I use now That is, I think, the big problem. Just using pip has never really worked so people have built tools on top of pip, in parallel with pip and replacements for pip. Basically there is no one way to do things.
    - sametmax 7y ago
      npm, yarn, bower, jamjs, pnpm... Besides, I have 15 years of Python behind me, I code and train in Python for a living, and I still use just pip. pip and venv are packaged with recent versions of Python and do the job fine.
  - bmn__ 7y ago
    Tests are not automatically run and centrally reported as part of the installation process. There is no good heuristic for picking one package over the other when they occupy a similar problem space. This is mostly a cultural problem, the community's efforts are lackluster. virtualenv is not installed by default, making bootstrapping into a separate prefix that is independent from the system installation unnecessarily aggravating. Semiautomatic packaging tools (e.g. pypi into rpm) produce low kwalitee packages and manual intervention more often needed compared to similar languages. Worse are languages that simply don't have much manpower behind them in absolute numbers, e.g. CL/quicklisp. Given Python's mindshare, the results are subpar.
    - 7y ago
- motles 7y ago
  "Amazingly well" has become 500MB node_modules directories in your source tree?
  - scrollaway 7y ago
    You should take a look at your virtualenvs once in a while. They're not particularly smaller and keep in mind that python has a much larger stdlib.
  - wongarsu 7y ago
    When I install the first 2-3 dependencies that causes npm to install 200 packages. I don't particularly like that, but it could be argued that this is a testament to how easy npm makes using third-party packages and how painlessly it does recursive dependency resolution.
    - carlmr 7y ago
      Exactly. It is such an easy to use package manager that people barely think about adding dependencies.
- sametmax 7y ago
  npm is worse IMO. First it forces the node_module in your repository, which mean you have to configure tons of tools to ignore it or you'll have a bad time. Then there is no way to have comfortable local command, so plenty of time you gotta have sudo -g. Recent npm version now have a tool for that but the experience is meh. Plus there is no builtin tool to manage several versions of node, so you gotta add nvm on top of that. And then if you work on the front end, you can't even use whatever you npm installed stuff, you gotta setup the entire webpack shenaigan, making it literally the hardest stuff to setup. It's so complicated we rely on black box such as create-react-app to do the job for us.
  - jononor 7y ago
    > Then there is no way to have comfortable local command, so plenty of time you gotta have sudo -g. Recent npm version now have a tool for that but the experience is meh. export PATH=$PATH:./node_modules/.bin/ or put the command into `scripts` of package.json, and use `npm run-script foo`. Have never used npm -g, and never will. Global packages are owned by the system package manager.
    - sametmax 7y ago
      Local commands doesn't mean you have a project. It can be a general utility.
stackzero 7y ago
The problem is there's no clear way to get started with setting up a python package. Every other day someone ask a question on SO about it and the first google result is outdated distutils IIRC
- d0mine 7y ago
  Have you looked at https://packaging.python.org/ https://packaging.python.org/
sametmax 7y ago
It's actually pretty ok now, espacially with .whl for pretty much everything avoiding compilation and venv being provided with recent Python. The problem is the sheer amount of bad information you find online, so that people take hours figuring how things work. E.G: - people telling you to sudo pip install - people telling you to use virtualenv instead of venv - people not telling you to do python -m - people not telling you about setup.cfg
- pas 7y ago
  Pipenv!
  - sametmax 7y ago
    That is one of the bad information: people that don't know how to use pip and venv shouldn't use pipenv, since you need pio to install it properly, and it will manage venv. Don't run before you know how to walk.
    - pas 7y ago
      no. you don't you need python (and the distutils module) and you can use the get-pipenv.py and separating venv management from pip is what causes The Mess

by jl6 7y ago

carlmr 7y ago

No, it's still one of the worst package management systems out there. I'm always wondering if it has to do with the language and or just the package manager itself. Rust's cargo, Node's npm and probably quite a few others exist that work amazingly well.

alexchamberlain 7y ago
What makes it the worst? Is it non-pure Python extensions, or is there something missing for pure python packages too?
- lmm 7y ago
  It's broken for pure python too. The standards were too heavily influenced by traditional unix sysadmins who love server-global libraries, so that's still the direction all the tooling guides you in despite being the worst way to work with Python. Every couple of years they rewrite it without fixing any of the problems. Sometimes they make it worse, as with e.g. moving pip into python. I struggle to imagine the decision-making process that could come up with these outcomes - surely they must be studying what works for other languages or there'd be no way they'd manage to avoid it all so perfectly.
  - sametmax 7y ago
    Not anymore, .whl files are now similar to npm packages, with static metadata and just unpacked on pip installed. Most popular packages are .whl now.
bow_ 7y ago
Why do you think it is the worst? If you don't consider Python, what would you then think is the worst? Genuinely curious. I use Python daily and I rarely encounter problems with it. There is some getting used to in the beginning, but I went through a similar phase when I started using npm and cargo as well. What I can say is that I had to go through a lot of experimentation myself to arrive at the tools I use now (pyenv + Poetry). And if anything, maybe the lack of one way that is adopted by everyone in the community is the problem.
- dagw 7y ago
  What I can say is that I had to go through a lot of experimentation myself to arrive at the tools I use now That is, I think, the big problem. Just using pip has never really worked so people have built tools on top of pip, in parallel with pip and replacements for pip. Basically there is no one way to do things.
  - sametmax 7y ago
    npm, yarn, bower, jamjs, pnpm... Besides, I have 15 years of Python behind me, I code and train in Python for a living, and I still use just pip. pip and venv are packaged with recent versions of Python and do the job fine.
- bmn__ 7y ago
  Tests are not automatically run and centrally reported as part of the installation process. There is no good heuristic for picking one package over the other when they occupy a similar problem space. This is mostly a cultural problem, the community's efforts are lackluster. virtualenv is not installed by default, making bootstrapping into a separate prefix that is independent from the system installation unnecessarily aggravating. Semiautomatic packaging tools (e.g. pypi into rpm) produce low kwalitee packages and manual intervention more often needed compared to similar languages. Worse are languages that simply don't have much manpower behind them in absolute numbers, e.g. CL/quicklisp. Given Python's mindshare, the results are subpar.
  - 7y ago
motles 7y ago
"Amazingly well" has become 500MB node_modules directories in your source tree?
- scrollaway 7y ago
  You should take a look at your virtualenvs once in a while. They're not particularly smaller and keep in mind that python has a much larger stdlib.
- wongarsu 7y ago
  When I install the first 2-3 dependencies that causes npm to install 200 packages. I don't particularly like that, but it could be argued that this is a testament to how easy npm makes using third-party packages and how painlessly it does recursive dependency resolution.
  - carlmr 7y ago
    Exactly. It is such an easy to use package manager that people barely think about adding dependencies.
sametmax 7y ago
npm is worse IMO. First it forces the node_module in your repository, which mean you have to configure tons of tools to ignore it or you'll have a bad time. Then there is no way to have comfortable local command, so plenty of time you gotta have sudo -g. Recent npm version now have a tool for that but the experience is meh. Plus there is no builtin tool to manage several versions of node, so you gotta add nvm on top of that. And then if you work on the front end, you can't even use whatever you npm installed stuff, you gotta setup the entire webpack shenaigan, making it literally the hardest stuff to setup. It's so complicated we rely on black box such as create-react-app to do the job for us.
- jononor 7y ago
  > Then there is no way to have comfortable local command, so plenty of time you gotta have sudo -g. Recent npm version now have a tool for that but the experience is meh. export PATH=$PATH:./node_modules/.bin/ or put the command into `scripts` of package.json, and use `npm run-script foo`. Have never used npm -g, and never will. Global packages are owned by the system package manager.
  - sametmax 7y ago
    Local commands doesn't mean you have a project. It can be a general utility.

stackzero 7y ago

The problem is there's no clear way to get started with setting up a python package. Every other day someone ask a question on SO about it and the first google result is outdated distutils IIRC

d0mine 7y ago
Have you looked at https://packaging.python.org/ https://packaging.python.org/

sametmax 7y ago

It's actually pretty ok now, espacially with .whl for pretty much everything avoiding compilation and venv being provided with recent Python. The problem is the sheer amount of bad information you find online, so that people take hours figuring how things work. E.G: - people telling you to sudo pip install - people telling you to use virtualenv instead of venv - people not telling you to do python -m - people not telling you about setup.cfg

pas 7y ago
Pipenv!
- sametmax 7y ago
  That is one of the bad information: people that don't know how to use pip and venv shouldn't use pipenv, since you need pio to install it properly, and it will manage venv. Don't run before you know how to walk.
  - pas 7y ago
    no. you don't you need python (and the distutils module) and you can use the get-pipenv.py and separating venv management from pip is what causes The Mess