3 ms·

Can't you resolve the dependencies by running pip download when you have internet and later serving that directory with a local HTTP server as the parent sugges

by vasco 4y ago

Can't you resolve the dependencies by running pip download when you have internet and later serving that directory with a local HTTP server as the parent suggested? Pip download will resolve all the dependencies for you already the same way as pip install would.

idop 4y ago
No, as I mention both in this post and in the README. Pip will download binary distributions (wheels) that were compiled for the system it is running on. If my mirror is meant to serve a different version of Python installed on a different OS with a different libc (or other such differences), then it won't work. I could try to match the target environment on the mirroring side, say with Docker, but this is either cumbersome or still not possible if you have legacy environments from years before.
- vasco 4y ago
  Makes sense and you're right we did encounter issues when changing platforms at the time when we were using some self rolled janky versio n of this! Thanks
- jpitz 4y ago
  By default it will download wheels for the system it is running on, but there are knobs to tweak that. https://pip.pypa.io/en/stable/cli/pip_download/#cmdoption-platform https://pip.pypa.io/en/stable/cli/pip_download/#cmdoption-pl... https://peps.python.org/pep-0425/ https://peps.python.org/pep-0425/
nijave 4y ago
You can download as source packages instead of wheels but then you need to make sure you have all the requisite compilers and libraries needed. This isn't an issue for Python-only dependencies but can be difficult for dependencies with lots of native code like numpy/pandas where you need a C toolchain & Fortran toolchain installed (and possibly other libs) If you're using something like Docker/containers, you can download the dependencies inside the container and be reasonably sure you get the right wheels. This becomes trickier when you have different setups like developers on Windows and production on Linux.

by vasco 4y ago

idop 4y ago

No, as I mention both in this post and in the README. Pip will download binary distributions (wheels) that were compiled for the system it is running on. If my mirror is meant to serve a different version of Python installed on a different OS with a different libc (or other such differences), then it won't work. I could try to match the target environment on the mirroring side, say with Docker, but this is either cumbersome or still not possible if you have legacy environments from years before.

vasco 4y ago
Makes sense and you're right we did encounter issues when changing platforms at the time when we were using some self rolled janky versio n of this! Thanks
jpitz 4y ago
By default it will download wheels for the system it is running on, but there are knobs to tweak that. https://pip.pypa.io/en/stable/cli/pip_download/#cmdoption-platform https://pip.pypa.io/en/stable/cli/pip_download/#cmdoption-pl... https://peps.python.org/pep-0425/ https://peps.python.org/pep-0425/

nijave 4y ago

You can download as source packages instead of wheels but then you need to make sure you have all the requisite compilers and libraries needed. This isn't an issue for Python-only dependencies but can be difficult for dependencies with lots of native code like numpy/pandas where you need a C toolchain & Fortran toolchain installed (and possibly other libs) If you're using something like Docker/containers, you can download the dependencies inside the container and be reasonably sure you get the right wheels. This becomes trickier when you have different setups like developers on Windows and production on Linux.