3 ms·
It's important to understand that this issue was specifically caused by a deficiency in Python's packaging system: dependencies are specified by name and versio
by KerrickStaley 4y ago
It's important to understand that this issue was specifically caused by a deficiency in Python's packaging system: dependencies are specified by name and version, and if the user configures multiple package repositories, there's no way to control which repository they get the package from. So if a package is normally only available on PyTorch's nightly package index, someone can upload the same package name and version on PyPI and it will take precedence.
It would be better if packages could specify a cryptographic hashes of the packages they depend on and/or specify the package repo that a package should come from. There is some prior art for this in e.g. the Javascript and Rust ecosystems.
This problem did not occur simply because pytorch has many dependencies and one of them got compromised. If pip allowed you to specify dependencies in a more secure way, this would not have happened.
- Noumenon72 4y agoWould it work to make a requirements-pytorch.txt that doesn't look in PyPi? --index-url https://user:pass@path/to/pytorch/nightly/repo
- codedokode 4y agoThis can be solved by using namespaced names for packages from third-party repositories, e.g. "pytorch/triton" instead of just "triton". In this case package from one repo cannot replace another repo's package.
- Too 4y agoIt is possible to specify hashes. https://pip.pypa.io/en/stable/topics/secure-installs/ https://pip.pypa.io/en/stable/topics/secure-installs/
- shanipribadi 4y agohashes would also need to be specified for all dependencies (transitives) in case they were needed, and all dependencies need to be pinned to specific versions as well. hence this would only work when users are making use of venvs, instead of user install / site install setup.