3 ms·

How do you lock all dependencies to a hash? Let's says if one of packages that you depend on decided to depend on some third package. The maintainer of that th

How do you lock all dependencies to a hash? Let's says if one of packages that you depend on decided to depend on some third package. The maintainer of that third package decides to stop maintaining it and gives it to someone else. This someone else decides to inject bitcoin mining into the third package. How will you know with pip that you are not pulling malicious code into production?

disgruntledphd2 6y ago
How do you handle C/C++ dependencies? This is super common in the DS/ML space, and until recently, pip was actively unhelpful (i see you have numpy version x, let's install version y silently because you definitely hand review all the transitive dependencies before installing a new package).
- Isinlor 6y ago
  In that case building a container image and testing it before deployment seems to be the easiest solution, certainly not ideal. Otherwise, I have no idea how to handle cross language dependencies well. My guess would be that operating system package managers ought to handle it, but I never tried to work with something like Debian repositories. Judging by what I've heard from other people it's a lot of pain.
- Doxin 6y ago
  These days ideally those dependencies would ship a wheel which solves the problem. If they don't you are expected to have a working C/C++ compiler set up which is definitely not ideal. At least on linux it's often as simple as installing a build-essentials package. No clue what that'd be like on other OSes since I don't use them.
  - disgruntledphd2 6y ago
    Only if the wheels aren't incompatible. I think my issues are really with pip and it's dependency resolver. Give me conda every day, it may take a while but it handles compiled dependencies and doesn't break my environment without at least telling me. And yes, I use docker containers now.
Doxin 6y ago
Well presumably under that threat model you'd run your own package cache. But in practice that doesn't really seem to be a thing that happens. Python hasn't had the equivalent of left-pad yet as far as I know.
- Isinlor 6y ago
  You may want to take a look: https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610 https://medium.com/@alex.birsan/dependency-confusion-4a5d60f... Security by "it hasn't happen yet" is not a security.

disgruntledphd2 6y ago

How do you handle C/C++ dependencies? This is super common in the DS/ML space, and until recently, pip was actively unhelpful (i see you have numpy version x, let's install version y silently because you definitely hand review all the transitive dependencies before installing a new package).

Isinlor 6y ago
In that case building a container image and testing it before deployment seems to be the easiest solution, certainly not ideal. Otherwise, I have no idea how to handle cross language dependencies well. My guess would be that operating system package managers ought to handle it, but I never tried to work with something like Debian repositories. Judging by what I've heard from other people it's a lot of pain.
Doxin 6y ago
These days ideally those dependencies would ship a wheel which solves the problem. If they don't you are expected to have a working C/C++ compiler set up which is definitely not ideal. At least on linux it's often as simple as installing a build-essentials package. No clue what that'd be like on other OSes since I don't use them.
- disgruntledphd2 6y ago
  Only if the wheels aren't incompatible. I think my issues are really with pip and it's dependency resolver. Give me conda every day, it may take a while but it handles compiled dependencies and doesn't break my environment without at least telling me. And yes, I use docker containers now.

Doxin 6y ago

Well presumably under that threat model you'd run your own package cache. But in practice that doesn't really seem to be a thing that happens. Python hasn't had the equivalent of left-pad yet as far as I know.

Isinlor 6y ago
You may want to take a look: https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610 https://medium.com/@alex.birsan/dependency-confusion-4a5d60f... Security by "it hasn't happen yet" is not a security.