5 ms·
I don't think pip is to blame for that. PyTorch is sadly an enormous space hog. I just started a fresh virtual environment with "python -m venv venv" - running
by simonw 2y ago
I don't think pip is to blame for that. PyTorch is sadly an enormous space hog.
I just started a fresh virtual environment with "python -m venv venv" - running "du -h" showed it to be 21MB. After running "venv/bin/pip install torch" it's now 431MB.
The largest file in there is this one:
178M ./lib/python3.10/site-packages/torch/lib/libtorch_cpu.dylib
There's a whole section of the uv manual dedicated just to PyTorch: https://docs.astral.sh/uv/guides/integration/pytorch/ https://docs.astral.sh/uv/guides/integration/pytorch/
(I just used find to locate as many libtorch_cpu.dylib files as possible on my laptop and deleted 5.5GB of them)
- sieve 2y agoI use uv pip to install dependencies for any LLM software I run. I am not sure if uv re-implements the pip logic or hands over resolution to pip. But it does not change the fact that I have multiple versions of torch + multiple installations of the same version of torch in the cache. Compare this to the way something like maven/gradle handles this and you have to wonder WTF is going on here.
- simonw 2y agouv implements its own resolution logic independently of pip. Maybe your various LLM libraries are pinning different versions of Torch? Different Python versions each need their own separate Torch binaries as well. At least with uv you don't end up with separate duplicate copies of PyTorch in each of the virtual environments for each of your different projects!
- sieve 2y ago> Different Python versions each need their own separate Torch binaries as well Found this the hard way. Something to do with breakage in ABI perhaps. Was looking at the way python implements extensions the other day. Very weird.
- woodruffw 2y ago> Something to do with breakage in ABI perhaps. Was looking at the way python implements extensions the other day. Very weird. Yes, it's essentially that: CPython doesn't guarantee exact ABI stability between versions unless the extension (and its enclosing package) intentionally build against the stable ABI[1]. The courteous thing to do in the Python packaging ecosystem is to build "abi3" wheels that are stable and therefore don't need to be duplicated as many times (either on the index or on the installing client). Torch doesn't build these wheels for whatever reason, so you end up with multiple slightly different but functionally identical builds for each version of Python you're using. TL;DR: This happens because of an interaction between two patterns that Python makes very easy: using multiple Python versions, and building/installing binary extensions. In a sense, it's a symptom of Python's success: other ecosystems don't have these problems because they have far fewer people running multiple configurations simultaneously. [1]: https://docs.python.org/3/c-api/stable.html https://docs.python.org/3/c-api/stable.html
- sieve 2y agoMy use of python is somewhat recent. But the two languages that I have used a lot of - Java and JS - have interpreters that were heavily optimized over time. I wonder why that never happened with python and, instead, everyone continues to write their critical code in C/Rust. I am planning to shift some of my stuff to pypy (so a "fast" python exists, kind of). But some dependencies can be problematic, I have heard.
- Numerlor 2y agopython just didn't have much momentum until relatively recently, despite it's age. There are efforts to speed it up going on now backed by Microsoft. For pypy it's in a weird spot as the things it does fast are the ones you'd usually just offload to a module implemented in C
- sieve 2y ago> There are efforts to speed it up Well, the extensions are going to complicate this a lot. A fast JIT interpreter cannot reach into a library and do its magic there the way HotSpot/V8 can with native Java/JS code.
- conradev 2y agouv should hard link files if they’re identical like Nix does If a package manager stores more than it needs to, it is a package manager problem.
- d0mine 2y agoWhat makes you think it doesn't?
- conradev 2y agoThis from above: > I just used find to locate as many libtorch_cpu.dylib files as possible on my laptop and deleted 5.5GB of them but maybe it wasn’t actually 5.5 GB!
- rfoo 2y agoIt was like a dozen different versions of libtorch_cpu.dylib. So hardlink does not help.
- rat87 2y agoIt probably is 5.5GB. UV caches versions of packages and symlinks to them but pytorch is infamous for many different versions especially for different features. And since its compiled dependency even if UV was to attempt the the more complicated method of symlinking to use a single version of identical files it wouldn't help much. You'd probably need to store binary diffs or chunks of files that are binary identical, at that point your code would probably start to resemble a file system in user space and time to switch to a particular version of the files (ie create thek as qctual files in filesystem) would be much higher. Also I believe uvs cache is separate from the pip cache so you could have different copies in both. I think there's a uv cache prune command. Arguably it should offer to install a from job to do it periodically
- Spivak 2y agoYou're about to be pleasantly surprised then. https://docs.astral.sh/uv/reference/settings/#link-mode https://docs.astral.sh/uv/reference/settings/#link-mode It's even the default. Here's where it's implemented if you're curious https://github.com/astral-sh/uv/blob/f394f7245377b6368b9412d88575446602f22525/crates/uv-install-wheel/src/linker.rs#L280 https://github.com/astral-sh/uv/blob/f394f7245377b6368b9412d...
- nighthawk454 2y agoTesting this out locally, repro'd pretty similar numbers on macOS ARM and docker. Unfortunately the CPU-only build isn't really any smaller either, thought it was