3 ms·

I wonder if the target of loglib-modules and hkg-sol-utils is a specific company running its own internal PyPi server, or something like a scientific collaborat

by hcrean 4y ago

I wonder if the target of loglib-modules and hkg-sol-utils is a specific company running its own internal PyPi server, or something like a scientific collaboration (which can be thousands of people and billions of dollars, especially in the drugs discovery domain). PIP is designed to treat all indexes as mirrors, rather than to specify the source of a package: So a higher package version of an internal package name would be chosen whether it is on public or private PyPi. Equally a certain percentage of the time it would choose public over private with the same package version.

legutierr 4y ago
Wow. So what you are saying is that someone with knowledge of a company’s internal private package dependencies would be able to hijack their build process by publishing a higher-numbered version of the same package to the public PyPi? And even if the build system explicitly references the package version number, part of the time the package would be taken from the public PyPi instead of the company’s own package server? How do you mitigate this, other than just totally blocking access to the public PyPi? Shouldn’t everyone be blocking access to PyPi, then, and only be relying on their own private package server?
- hcrean 4y ago
  So the correct solution is really to pre-approve dependencies and store them in a local PyPi. Defence, Medical and some other high-sec industries do this as a matter of course. The problem with this approach is the amount of security toil required to vet packages is large, generally too large for most companies to buy into. So one solution I see in the wild a lot is to prefix your company packages with a company identifier, then to set proxy rules to prevent fetching these from public PyPi (often these are set directly on the private PyPi and all traffic is directed through it). Having been an agency pentester, I can tell you that given access to one company package (Open Sourced?), you can correctly guess the package structure, naming conventions and internal build infra of the company 90+% of the time.
- di 4y ago
  Using hash checking mode: https://pip.pypa.io/en/stable/topics/secure-installs/#hash-checking-mode https://pip.pypa.io/en/stable/topics/secure-installs/#hash-c...
  - legutierr 4y ago
    Straight from the horse's mouth! Thank you, I didn't know about this.
- salmon 4y ago
  This is an attack vector which has been exploited in the past to great effect[1]. [1] https://arstechnica.com/information-technology/2021/02/supply-chain-attack-that-fooled-apple-and-microsoft-is-attracting-copycats/ https://arstechnica.com/information-technology/2021/02/suppl...

by hcrean 4y ago

legutierr 4y ago

Wow. So what you are saying is that someone with knowledge of a company’s internal private package dependencies would be able to hijack their build process by publishing a higher-numbered version of the same package to the public PyPi? And even if the build system explicitly references the package version number, part of the time the package would be taken from the public PyPi instead of the company’s own package server? How do you mitigate this, other than just totally blocking access to the public PyPi? Shouldn’t everyone be blocking access to PyPi, then, and only be relying on their own private package server?

hcrean 4y ago
So the correct solution is really to pre-approve dependencies and store them in a local PyPi. Defence, Medical and some other high-sec industries do this as a matter of course. The problem with this approach is the amount of security toil required to vet packages is large, generally too large for most companies to buy into. So one solution I see in the wild a lot is to prefix your company packages with a company identifier, then to set proxy rules to prevent fetching these from public PyPi (often these are set directly on the private PyPi and all traffic is directed through it). Having been an agency pentester, I can tell you that given access to one company package (Open Sourced?), you can correctly guess the package structure, naming conventions and internal build infra of the company 90+% of the time.
di 4y ago
Using hash checking mode: https://pip.pypa.io/en/stable/topics/secure-installs/#hash-checking-mode https://pip.pypa.io/en/stable/topics/secure-installs/#hash-c...
- legutierr 4y ago
  Straight from the horse's mouth! Thank you, I didn't know about this.
salmon 4y ago
This is an attack vector which has been exploited in the past to great effect[1]. [1] https://arstechnica.com/information-technology/2021/02/supply-chain-attack-that-fooled-apple-and-microsoft-is-attracting-copycats/ https://arstechnica.com/information-technology/2021/02/suppl...