Have you considered using the copy-on-write features that some filesystems (btrfs) now support? For example, the lowly cp command now supports CoW via --reflink

Have you considered using the copy-on-write features that some filesystems (btrfs) now support? For example, the lowly cp command now supports CoW via --reflink=always Also, that ks for git-annex! I use it for hundreds of gigabytes of files, and find the one-copy-per-file support essential for how I use it.