3 ms·

I wrote a similar (but simpler) script which would replace a file by a hardlink if it has the same content. My main motivation was for the packages of Python v

I wrote a similar (but simpler) script which would replace a file by a hardlink if it has the same content. My main motivation was for the packages of Python virtual envs, where I often have similar packages installed, and even if versions are different, many files would still match. Some of the packages are quite huge, e.g. Numpy, PyTorch, TensorFlow, etc. I got quite some disk space savings from this. https://github.com/albertz/system-tools/blob/master/bin/merge-identical-files.py https://github.com/albertz/system-tools/blob/master/bin/merg...

andrewla 2y ago
This does not use hard links or symlinks; this uses a feature of the filesystem that allows the creation of copy-on-write clones. [1] [1] https://en.wikipedia.org/wiki/Apple_File_System#Clones https://en.wikipedia.org/wiki/Apple_File_System#Clones
- gurjeet 2y ago
  So albertzeyer's script can be adapted to use `cp -c` command, to achieve the same effect as Hyperspace.
  - pfranz 2y ago
    If you'd like. In the blog post he says he wrote the prototype in an afternoon. Hyperspace does try hard to preserve unique metadata as well as other protections.
notatallshaw 2y ago
uv does this out of the box, I think other tools (poetry, hatch, pdm, etc.) do as well but I have less experience with the details.