3 ms·
A shameless plug but it is a simple —and probably bad written— tool I made many years ago to scratch an itch, and I still use it. It finds duplicate files and
by andmarios 5y ago
A shameless plug but it is a simple —and probably bad written— tool I made many years ago to scratch an itch, and I still use it.
It finds duplicate files and replaces them with hard links, saving you space. Just make sure you provide it with paths in the same filesystem.
I originally wrote it to save some space from personal files (videos, photos, etc), but it turned out very useful for tar files, docker images, websites, and more.
For example I maintain a tar file and a docker image with Kafka connectors which share many jar files. Using duphard I can save hundreds of megabytes, or even more than a gigabyte! For a documentation website with many copies of the same image (let's just say some static generators favor this practice for maintaining multiple versions), I can reduce the website size by 60%+, which then makes ssh copies, docker pulls, etc way faster speeding up deployment times.
https://github.com/andmarios/duphard https://github.com/andmarios/duphard
- deleted 5y ago[deleted]
- coryrc 5y agoFdupes does that too
- andmarios 5y agoUnfortunately fdupes does not do this or at least nothing in their docs points to this functionality. I say unfortunately because before writing duphard, I tested fdupes and a couple other utilities (duff and duperemove) but none offered the functionality I needed.
- pkolaczk 5y agoA shameless plug: fclones does that as well, including support for symlinks.
- andmarios 5y agoIt looks good! When I wrote duphard, fclones didn't exist. :)