Git does delta compression on packfiles, and if the same data occurs in different in different files, it actually can deduplicate it, even if it's not at the sa

Git does delta compression on packfiles, and if the same data occurs in different in different files, it actually can deduplicate it, even if it's not at the same offset. Here's a demo. First, create two test files. The files both contain the same two 1-megabyte chunks of random bytes, but in the opposite order: $ openssl rand 1000000 > a $ openssl rand 1000000 > b $ cat a b > ab $ cat b a > ba $ du -sh ab ba 2.0M ab 2.0M ba Commit them to Git and see that it requires 4 MB to store these two 2 MB files: $ git init Initialized empty Git repository in /tmp/x/.git/ $ git add ab ba $ git commit -m 'add two files' [master (root-commit) 7f75af0] add two files 2 files changed, 0 insertions(+), 0 deletions(-) create mode 100644 ab create mode 100644 ba $ du -sh .git 4.0M .git Run garbage collection, which creates a packfile. Note "delta 1" and note that disk usage dropped to a bit over the size of one of the files. $ git gc Enumerating objects: 4, done. Counting objects: 100% (4/4), done. Delta compression using up to 6 threads Compressing objects: 100% (4/4), done. Writing objects: 100% (4/4), done. Total 4 (delta 1), reused 0 (delta 0), pack-reused 0 $ du -sh .git 2.1M .git I'm not sure as if it's as sophisticated as some backup tools, though.