3 ms·

There is not a conflict here. PyTorch depends on many packages and none of them were compromised. Instead, a limitation with the way pip installs dependencies a

There is not a conflict here. PyTorch depends on many packages and none of them were compromised. Instead, a limitation with the way pip installs dependencies allowed someone to create a package on PyPI with the same name and version as the one on PyTorch's nightly package index, which took precedence over the real package. This could be fixed by supporting better ways to specify Python dependencies in pip, without any dampening effect on the package ecosystem.

theteapot 4y ago
I think blame is shared between pip being ultra vulnerable to foot guns like this and whoever put together the PyTorch nightly install not seeing the dep confusion issue from a mile away.
shanipribadi 4y ago
there's a few factors here that allow dependency confusion attack to happen here 1. pytorch publishes their nightly package on their repo which depends on a custom triton build provided on their repo, but using a package name they don't own in pypi. This has been mitigated by them by renaming the dependency from torchtriton to pytorch-triton, reserving pytorch-triton package in pypi, and changing the dependency name on the newer nightly builds 20221231 forward to point to pytorch-triton package instead. 2. pytorch installation instruction for nightly using pip in (https://pytorch.org/get-started/locally/#start-locally https://pytorch.org/get-started/locally/#start-locally), make use of the --extra-index-url option. This is a known vector of dependency confusion attacks, and is an inherently insecure method of installing packages from private repositories. The recommended approach of distributing wheels in private repositories is by using a repository server that allows proxying/redirecting the public pypi packages to pypi, and users should be using a single --index-url pointed to that private repository (assuming the maintainer of that private repository is to be trusted). --extra-index-url is meant to provide mirror urls (serving the same set of packages as the main one), rather than to combine repos with different sets of packages.
- codethief 4y ago
  > The recommended approach of distributing wheels in private repositories is by using a repository server that allows proxying/redirecting the public pypi packages to pypi, and users should be using a single --index-url pointed to that private repository (assuming the maintainer of that private repository is to be trusted). Alternatively, keep a separate requirements_private.txt around for private dependencies and add a line --index-url <my private repository>.

theteapot 4y ago

I think blame is shared between pip being ultra vulnerable to foot guns like this and whoever put together the PyTorch nightly install not seeing the dep confusion issue from a mile away.

shanipribadi 4y ago

there's a few factors here that allow dependency confusion attack to happen here 1. pytorch publishes their nightly package on their repo which depends on a custom triton build provided on their repo, but using a package name they don't own in pypi. This has been mitigated by them by renaming the dependency from torchtriton to pytorch-triton, reserving pytorch-triton package in pypi, and changing the dependency name on the newer nightly builds 20221231 forward to point to pytorch-triton package instead. 2. pytorch installation instruction for nightly using pip in (https://pytorch.org/get-started/locally/#start-locally https://pytorch.org/get-started/locally/#start-locally), make use of the --extra-index-url option. This is a known vector of dependency confusion attacks, and is an inherently insecure method of installing packages from private repositories. The recommended approach of distributing wheels in private repositories is by using a repository server that allows proxying/redirecting the public pypi packages to pypi, and users should be using a single --index-url pointed to that private repository (assuming the maintainer of that private repository is to be trusted). --extra-index-url is meant to provide mirror urls (serving the same set of packages as the main one), rather than to combine repos with different sets of packages.

codethief 4y ago
> The recommended approach of distributing wheels in private repositories is by using a repository server that allows proxying/redirecting the public pypi packages to pypi, and users should be using a single --index-url pointed to that private repository (assuming the maintainer of that private repository is to be trusted). Alternatively, keep a separate requirements_private.txt around for private dependencies and add a line --index-url <my private repository>.