4 ms·

As long as you don't have lots of C or Cpp dependencies, Python packaging is apparently totally fine. Unfortunately I don't live in that world :(

acdha 6y ago
I mean, I do use a lot of C code and the things which break are external C level dependencies. You can blame Python for those but it won’t help fix them.
- disgruntledphd2 6y ago
  Well when pip installs an incompatible version of Numpy relative to my current setup (without warning me) then I do indeed blame pip. To be fair, this release makes that much better, which I am extremely grateful for.
  - acdha 6y ago
    If a library doesn’t specify version compatibility and you don’t pin it but do choose to run an upgrade, why is that the installer’s problem? It’s like complaining about Toyota when you get a parking ticket.
    - disgruntledphd2 6y ago
      See, this isn't necessary in other languages. All I want is an error, so that I can assess what the right thing to do is. Honestly, I sometimes feel that people have stockholm syndrome with python packaging. Nonetheless, I'm really happy about this release, as it will help to prevent these problems. It's also worth noting that I didn't even learn Python for a long time because I kept breaking my install while trying to get started, so this stuff has real costs in terms of adoption.
      - acdha 6y ago
        The same problems happen in those languages — the underlying problem is someone not specifying version constraints they actually have. If you get more experience with those other languages you'll definitely encounter that with projects which aren't carefully following semantic versioning. In particular, this goes back to your previous confusion about C/C++ — if you're having a problem with those toolchains, the underlying problem is that they don't have a package manager at all and every downstream tool has to compensate for that. The one part which is worse in Python than some of those other languages is that the default tool doesn't track all of the dependencies when you install a new package. If you're not using something like pipenv, poetry, etc. you can add "foo" to your requirements file and have it work now but break in the future if you aren't using a tool to pin specific versions (e.g. pip-tools). Some other tools like NPM or Cargo avoid that by default but that's certainly not true of all languages and it's something which is widely acknowledged as an area for improvement by the people recommending or working on the aforementioned tools.
        disgruntledphd2 6y ago
        So, my major comparator for ease of installation is a statistical programming language called R. Both Python and R have equal numbers of C/C++ dependencies, but the only breakage I experience is with Python. This is for a number of reasons: 1) R has CRAN (like CPAN, but for R). If your package does not build on the latest version of R, it becomes unavailable. 2) if your package has C++ dependencies, the installation will error out and say which headers its missing. This means that I can search for the header and install it. 3) Because of the removal of obsolete packages, dependencies just work and you can be assured that a package will build with a given version of R. Updates can be a problem, but they are opt-in at the package level. The core issue with Python is the acceptance that different projects can have entirely different dependencies, which means that this breakage is relatively common. Like, I build C/C++ on a semi-regular basis, and it can be super-painful, but Python is the only higher-level language that gives me this heartache. Not to mention that if you install pip from the Linux repositories, you're setting yourself up for a world of pain (this actually stopped me from learning Python for a number of years). To be fair, I mostly use conda for C/C++ dependencies as it works much better, but lots of packages are still only available through pip, which means I can't avoid the problem. To reiterate, I am super happy with this release of pip, as it will make my life much better. It doesn't go far enough, but it is much, much better than what existed before. Finally, Python packaging is a horrible wart on an otherwise good language, and I'm probably going to argue against using it in the future because of this.
Nextgrid 6y ago
Dependency management in C or C++ is always an absolute shit-show.
- stingraycharles 6y ago
  The only way to do it and to stay sane is to just include your dependencies in your source tree.

acdha 6y ago

I mean, I do use a lot of C code and the things which break are external C level dependencies. You can blame Python for those but it won’t help fix them.

disgruntledphd2 6y ago
Well when pip installs an incompatible version of Numpy relative to my current setup (without warning me) then I do indeed blame pip. To be fair, this release makes that much better, which I am extremely grateful for.
- acdha 6y ago
  If a library doesn’t specify version compatibility and you don’t pin it but do choose to run an upgrade, why is that the installer’s problem? It’s like complaining about Toyota when you get a parking ticket.
  - disgruntledphd2 6y ago
    See, this isn't necessary in other languages. All I want is an error, so that I can assess what the right thing to do is. Honestly, I sometimes feel that people have stockholm syndrome with python packaging. Nonetheless, I'm really happy about this release, as it will help to prevent these problems. It's also worth noting that I didn't even learn Python for a long time because I kept breaking my install while trying to get started, so this stuff has real costs in terms of adoption.
    - acdha 6y ago
      The same problems happen in those languages — the underlying problem is someone not specifying version constraints they actually have. If you get more experience with those other languages you'll definitely encounter that with projects which aren't carefully following semantic versioning. In particular, this goes back to your previous confusion about C/C++ — if you're having a problem with those toolchains, the underlying problem is that they don't have a package manager at all and every downstream tool has to compensate for that. The one part which is worse in Python than some of those other languages is that the default tool doesn't track all of the dependencies when you install a new package. If you're not using something like pipenv, poetry, etc. you can add "foo" to your requirements file and have it work now but break in the future if you aren't using a tool to pin specific versions (e.g. pip-tools). Some other tools like NPM or Cargo avoid that by default but that's certainly not true of all languages and it's something which is widely acknowledged as an area for improvement by the people recommending or working on the aforementioned tools.
      - disgruntledphd2 6y ago
        So, my major comparator for ease of installation is a statistical programming language called R. Both Python and R have equal numbers of C/C++ dependencies, but the only breakage I experience is with Python. This is for a number of reasons: 1) R has CRAN (like CPAN, but for R). If your package does not build on the latest version of R, it becomes unavailable. 2) if your package has C++ dependencies, the installation will error out and say which headers its missing. This means that I can search for the header and install it. 3) Because of the removal of obsolete packages, dependencies just work and you can be assured that a package will build with a given version of R. Updates can be a problem, but they are opt-in at the package level. The core issue with Python is the acceptance that different projects can have entirely different dependencies, which means that this breakage is relatively common. Like, I build C/C++ on a semi-regular basis, and it can be super-painful, but Python is the only higher-level language that gives me this heartache. Not to mention that if you install pip from the Linux repositories, you're setting yourself up for a world of pain (this actually stopped me from learning Python for a number of years). To be fair, I mostly use conda for C/C++ dependencies as it works much better, but lots of packages are still only available through pip, which means I can't avoid the problem. To reiterate, I am super happy with this release of pip, as it will make my life much better. It doesn't go far enough, but it is much, much better than what existed before. Finally, Python packaging is a horrible wart on an otherwise good language, and I'm probably going to argue against using it in the future because of this.

Nextgrid 6y ago

Dependency management in C or C++ is always an absolute shit-show.

stingraycharles 6y ago
The only way to do it and to stay sane is to just include your dependencies in your source tree.