6 ms·

(Genuine question). I only occasionally write python, but I just use venv and install requirements file. What toolchain challenges are out there for python?

hylaride 2y ago
For a large enough project, the dependency conflicts can get extremely frustrating, especially when it's time to update them. You may need to upgrade a dependency for security reasons cough cough requests cough cough, but some other dependency that calls it has pinned another version (range).
- grandempire 2y ago
  Yeah that is a nightmare. But isn’t that a problem on all package systems except more dynamic runtimes like NPM which can load many copies of the same library?
  - hylaride 2y ago
    It's a problem all languages have, but some are better at sorting it out. The way NPM does it solves one issue, but causes others. The big issue, IMHO, is that when you're dealing with interpreted languages it's very hard to lock down issues before runtime. With compiled or statically typed languages you tend to know a lot sooner where issues lie. I've had to update requests to deal with certificate issues (to support more modern ciphers/hashes etc) but I won't know until runtime if it even works.
    - grandempire 2y ago
      Agreed. I am not saying NPM is better, just that it side steps dependency resolution problems through its runtime. (I do not want 20 copies of the same library in my process)
- bigtimesink 2y ago
  Dependency conflicts become an issue for large projects in any language. It's less of a problem when the language's runtime is feature-rich since libraries will be less likely to use a third-party HTTP client. You can choose libraries with fewer dependencies, but that only gets you so far. At some point, you can put the libraries in you monorepo, but upgrades come with a large cost.
SatvikBeri 2y ago
Everywhere I've worked, I've had a few cases where we updated some dependencies on machine A (e.g. a developer's macbook), everything ran fine, we did the same updates on machine B (e.g. an Ubuntu EC2 instance) and everything broke. This is especially case with the numpy/scipy/pandas/etc. ecosystem. In one case this took days to fix, which is insane. I haven't had that experience with any other language. It's worth noting that all of these involved anaconda, which was the recommended way to install numeric libraries at the time. Other package managers might be better.
nyarlathotep_ 2y ago
Native dependencies are a huge PITA depending on their nature and lack of cross-platform portability. This can and will also pop up on CI.
mybazongas 2y ago
Well, every six months, someone finally solves Python's distribution problem.

hylaride 2y ago

For a large enough project, the dependency conflicts can get extremely frustrating, especially when it's time to update them. You may need to upgrade a dependency for security reasons cough cough requests cough cough, but some other dependency that calls it has pinned another version (range).

grandempire 2y ago
Yeah that is a nightmare. But isn’t that a problem on all package systems except more dynamic runtimes like NPM which can load many copies of the same library?
- hylaride 2y ago
  It's a problem all languages have, but some are better at sorting it out. The way NPM does it solves one issue, but causes others. The big issue, IMHO, is that when you're dealing with interpreted languages it's very hard to lock down issues before runtime. With compiled or statically typed languages you tend to know a lot sooner where issues lie. I've had to update requests to deal with certificate issues (to support more modern ciphers/hashes etc) but I won't know until runtime if it even works.
  - grandempire 2y ago
    Agreed. I am not saying NPM is better, just that it side steps dependency resolution problems through its runtime. (I do not want 20 copies of the same library in my process)
bigtimesink 2y ago
Dependency conflicts become an issue for large projects in any language. It's less of a problem when the language's runtime is feature-rich since libraries will be less likely to use a third-party HTTP client. You can choose libraries with fewer dependencies, but that only gets you so far. At some point, you can put the libraries in you monorepo, but upgrades come with a large cost.

SatvikBeri 2y ago

Everywhere I've worked, I've had a few cases where we updated some dependencies on machine A (e.g. a developer's macbook), everything ran fine, we did the same updates on machine B (e.g. an Ubuntu EC2 instance) and everything broke. This is especially case with the numpy/scipy/pandas/etc. ecosystem. In one case this took days to fix, which is insane. I haven't had that experience with any other language. It's worth noting that all of these involved anaconda, which was the recommended way to install numeric libraries at the time. Other package managers might be better.

nyarlathotep_ 2y ago

Native dependencies are a huge PITA depending on their nature and lack of cross-platform portability. This can and will also pop up on CI.

mybazongas 2y ago

Well, every six months, someone finally solves Python's distribution problem.