7 ms·

At least in Python's case, I think people are much too hard on the packaging system. I don't think anyone has significant issues with pip itself as opposed to t

At least in Python's case, I think people are much too hard on the packaging system. I don't think anyone has significant issues with pip itself as opposed to the runtime it exists within. Following from this, I think that most back-end applications should try solve their messy runtime environment issues with some containerization. Java/C(++) however...

mschuster91 4y ago
> I don't think anyone has significant issues with pip itself as opposed to the runtime it exists within. I'm running macOS with Macports, and oh boy. For a long time, the aws and eb CLI tools had different package requirements or whatnot, and you couldn't install both of them simultaneously for whatever reason. Some stuff installs fine with the system Python, some stuff needs Macports for a specific Python version, some stuff needs root access to install itself... anything Python is a hot mess.
- robertlagrant 4y ago
  While I'm trained to use venvs for everything, it would be nicer to have a flat global list of dependencies in every version required of every package, and specify allowed versions in each project, which will use what you have and/or download what's missing.
- nine_k 4y ago
  System Python? Root access? Oh. That's a terrible, terrible practice. First, one never touches the system Python. It's there for OS-managed stuff and to run OS components. Then, every Python-based tool should be installed in a separate, unprivileged virtualenv, with executables symlinked where you find them appropriate (usually ~/bin or /usr/local/bin). If you find yourself ever issuing a `sudo pip install`, chances are million to one that you are doing a disservice to to yourself.
  - mschuster91 4y ago
    > First, one never touches the system Python. It's there for OS-managed stuff and to run OS components. Why would one want to not use what the system provides? Everything not provided by the OS is additional maintenance burden on myself. The 'nix world has managed just fine with OS-provided bash, perl and other runtime dependencies for decades. > Then, every Python-based tool should be installed in a separate, unprivileged virtualenv, with executables symlinked where you find them appropriate (usually ~/bin or /usr/local/bin). No one has time for all that. I'm not a Python developer - as a user I'm happy enough if it barely works. When an ecosystem requires messing around with completely separate instances of the runtime for each program, that's not a good sign for the quality of the ecosystem as a whole.
    - anthk 4y ago
      Then when your system installed Python begins to crap out your whole system down, don't ask for solutions.
      - vt85 4y ago
        [dead]
nicoburns 4y ago
> I don't think anyone has significant issues with pip itself as opposed to the runtime it exists within. Pip has lots of issues (IMO it's one of the worst package managers): - It installs dependencies globally by default! This makes it a massive pain to work with unless you get virtualenvs setup correctly. Which you have to remember to do every time you interact with a project. - It doesn't use lock files by default. You have to remember to lock your dependencies. - Packages that depend on C libraries have a tendency to fail with cryptic error messages if you don't have the library installed (which can be contrasted with node.js packages which tend to bundle the library, and will even compile it from source for you if there isn't a binary version available for your platform).
simiones 4y ago
Java's Maven has been the nicest package management tool I've ever worked with. Apart from the fact that definitions are a bit verbose, it supported everything I've ever wanted from such a tool: 1. Clear definitions of what is an artifact that it manages, easily described in a versioned file but still separate from your source code 2. Simple support to combine internal and external sources 3. Support for interacting with other package ecosystems (you can easily write a pom.xml for any other kind of package and reference it from your Java projects) 4. Not picky about version number formats 5. Not reliant on other tools to handle any step of package management (except for compilation, of course) 6. Works at the project level and has no problems about different versions being used in different projects on the same machine (with a shared cache for efficiency)
- groestl 4y ago
  And because of that, maven infrastructure is and has been the target of multiple build frameworks and languages.
- EFreethought 4y ago
  I would like to second that Maven works for me as well. A lot of posters have said that Rust and Go work well. Honestly, the OP sounds like the stereotype of a JS developer: "This is a problem for Javascript, therefore it must be a problem for everyone else, too." No, guys, it's just you. Every single time.
arcturus17 4y ago
I find myself in nightmarish scenarios with Python dependencies more often than I'd like to. This dependency is missing this wheel which is missing this bit of C tooling which needs system-level permissions to be installed through XCode on Mac (what?). I've somehow corrupted the package lock, now I'm looking at diffs and checking dependencies of dependencies, inspecting the virtual environment folder and questioning my life choices. PyCharm has somehow lost the sense of where the virtual environment is located, I now have a squiggly line and warning explosion in my IDE. You could argue many of these are not Python's fault, some can probably be attributed to carelessness on third-party and tooling devs or on my part. But let's invent a metric here: "% of time spent dependency troubleshooting over total programming time". I've had better experiences with this in other languages, and intuition tells me that my carelessness or that of third-party devs being equal, there is something not quite right about the Python dependency management story. I put up with it because like many others I think that the language is otherwise dope.
DemocracyFTW2 4y ago
> I don't think anyone has significant issues with pip and boy would you be wrong. I just tried to install `datasette-scraper` on my system. This didn't work because it would consistently pick up outdated versions of Django and `more-itertools` from the system's Python site-packages directory. By the way I learned to (1) install an updated version of Python that has the advantage of not being the one that the system uses; (2) do not use `pip` the executable but `python3.9 -m pip`; (3) use `pip` with argument `--target vendor` to install everything in a dedicated project-owned directory; (4) prefix my invocation of the software with `PYTHONPATH=./vendor` to add that to the import path; (5) add a `vendor/sitecustomize.py` file with the lines `import sys; sys.path.pop()` to remove the system's `site-packages` from the path. That's a local installation of a Python package in five easy steps, no 'virtual-env' or some such! Pip is great! /s
- Izkata 4y ago
  > no 'virtual-env' or some such! I'm pretty sure you're manually doing the same thing virtualenv (venv in python3) does for you...
  - DemocracyFTW2 4y ago
    I'd guess that, yes, and going forward I may take a 2nd look at virtualenv. That said, I am a little proud and glad I managed to take it this far in this particular instance because I feel it has improved my ability to deal with Pythonic installation gotchas. Pip is a great teaching tool! /s
  - __marvin_the__ 4y ago
    I have personally never had a problem with activating a virtualenv and `pip install`-ing inside it. Most libraries nowadays also specify minimum versions for packages (e.g., >=3.6, <4.0).
marcosdumay 4y ago
Everybody uses PIP, but the Python devs (well single BDFL) refuse to accept this and standardize the package manager. As a result, if you want to make a package, you must support everything. Also, PIP can't evolve, because nobody will make their packages even more complex to support PIP only idiosyncrasies. It doesn't matter that everybody uses PIP, you have to spend 90% of your time dealing with other package managers, so they are the ones on your mind all the time.

mschuster91 4y ago

> I don't think anyone has significant issues with pip itself as opposed to the runtime it exists within. I'm running macOS with Macports, and oh boy. For a long time, the aws and eb CLI tools had different package requirements or whatnot, and you couldn't install both of them simultaneously for whatever reason. Some stuff installs fine with the system Python, some stuff needs Macports for a specific Python version, some stuff needs root access to install itself... anything Python is a hot mess.

robertlagrant 4y ago
While I'm trained to use venvs for everything, it would be nicer to have a flat global list of dependencies in every version required of every package, and specify allowed versions in each project, which will use what you have and/or download what's missing.
nine_k 4y ago
System Python? Root access? Oh. That's a terrible, terrible practice. First, one never touches the system Python. It's there for OS-managed stuff and to run OS components. Then, every Python-based tool should be installed in a separate, unprivileged virtualenv, with executables symlinked where you find them appropriate (usually ~/bin or /usr/local/bin). If you find yourself ever issuing a `sudo pip install`, chances are million to one that you are doing a disservice to to yourself.
- mschuster91 4y ago
  > First, one never touches the system Python. It's there for OS-managed stuff and to run OS components. Why would one want to not use what the system provides? Everything not provided by the OS is additional maintenance burden on myself. The 'nix world has managed just fine with OS-provided bash, perl and other runtime dependencies for decades. > Then, every Python-based tool should be installed in a separate, unprivileged virtualenv, with executables symlinked where you find them appropriate (usually ~/bin or /usr/local/bin). No one has time for all that. I'm not a Python developer - as a user I'm happy enough if it barely works. When an ecosystem requires messing around with completely separate instances of the runtime for each program, that's not a good sign for the quality of the ecosystem as a whole.
  - anthk 4y ago
    Then when your system installed Python begins to crap out your whole system down, don't ask for solutions.
    - vt85 4y ago
      [dead]

nicoburns 4y ago

> I don't think anyone has significant issues with pip itself as opposed to the runtime it exists within. Pip has lots of issues (IMO it's one of the worst package managers): - It installs dependencies globally by default! This makes it a massive pain to work with unless you get virtualenvs setup correctly. Which you have to remember to do every time you interact with a project. - It doesn't use lock files by default. You have to remember to lock your dependencies. - Packages that depend on C libraries have a tendency to fail with cryptic error messages if you don't have the library installed (which can be contrasted with node.js packages which tend to bundle the library, and will even compile it from source for you if there isn't a binary version available for your platform).

simiones 4y ago

Java's Maven has been the nicest package management tool I've ever worked with. Apart from the fact that definitions are a bit verbose, it supported everything I've ever wanted from such a tool: 1. Clear definitions of what is an artifact that it manages, easily described in a versioned file but still separate from your source code 2. Simple support to combine internal and external sources 3. Support for interacting with other package ecosystems (you can easily write a pom.xml for any other kind of package and reference it from your Java projects) 4. Not picky about version number formats 5. Not reliant on other tools to handle any step of package management (except for compilation, of course) 6. Works at the project level and has no problems about different versions being used in different projects on the same machine (with a shared cache for efficiency)

groestl 4y ago
And because of that, maven infrastructure is and has been the target of multiple build frameworks and languages.
EFreethought 4y ago
I would like to second that Maven works for me as well. A lot of posters have said that Rust and Go work well. Honestly, the OP sounds like the stereotype of a JS developer: "This is a problem for Javascript, therefore it must be a problem for everyone else, too." No, guys, it's just you. Every single time.

arcturus17 4y ago

I find myself in nightmarish scenarios with Python dependencies more often than I'd like to. This dependency is missing this wheel which is missing this bit of C tooling which needs system-level permissions to be installed through XCode on Mac (what?). I've somehow corrupted the package lock, now I'm looking at diffs and checking dependencies of dependencies, inspecting the virtual environment folder and questioning my life choices. PyCharm has somehow lost the sense of where the virtual environment is located, I now have a squiggly line and warning explosion in my IDE. You could argue many of these are not Python's fault, some can probably be attributed to carelessness on third-party and tooling devs or on my part. But let's invent a metric here: "% of time spent dependency troubleshooting over total programming time". I've had better experiences with this in other languages, and intuition tells me that my carelessness or that of third-party devs being equal, there is something not quite right about the Python dependency management story. I put up with it because like many others I think that the language is otherwise dope.

DemocracyFTW2 4y ago

> I don't think anyone has significant issues with pip and boy would you be wrong. I just tried to install `datasette-scraper` on my system. This didn't work because it would consistently pick up outdated versions of Django and `more-itertools` from the system's Python site-packages directory. By the way I learned to (1) install an updated version of Python that has the advantage of not being the one that the system uses; (2) do not use `pip` the executable but `python3.9 -m pip`; (3) use `pip` with argument `--target vendor` to install everything in a dedicated project-owned directory; (4) prefix my invocation of the software with `PYTHONPATH=./vendor` to add that to the import path; (5) add a `vendor/sitecustomize.py` file with the lines `import sys; sys.path.pop()` to remove the system's `site-packages` from the path. That's a local installation of a Python package in five easy steps, no 'virtual-env' or some such! Pip is great! /s

Izkata 4y ago
> no 'virtual-env' or some such! I'm pretty sure you're manually doing the same thing virtualenv (venv in python3) does for you...
- DemocracyFTW2 4y ago
  I'd guess that, yes, and going forward I may take a 2nd look at virtualenv. That said, I am a little proud and glad I managed to take it this far in this particular instance because I feel it has improved my ability to deal with Pythonic installation gotchas. Pip is a great teaching tool! /s
- __marvin_the__ 4y ago
  I have personally never had a problem with activating a virtualenv and `pip install`-ing inside it. Most libraries nowadays also specify minimum versions for packages (e.g., >=3.6, <4.0).

marcosdumay 4y ago

Everybody uses PIP, but the Python devs (well single BDFL) refuse to accept this and standardize the package manager. As a result, if you want to make a package, you must support everything. Also, PIP can't evolve, because nobody will make their packages even more complex to support PIP only idiosyncrasies. It doesn't matter that everybody uses PIP, you have to spend 90% of your time dealing with other package managers, so they are the ones on your mind all the time.