Btrfs uses data and metadata checksums, those pre-existing checksums allow for checking for duplications without having to read the underlying files. So sounds

Btrfs uses data and metadata checksums, those pre-existing checksums allow for checking for duplications without having to read the underlying files. So sounds like we are violently agreeing. Yes you can deduplicate 1TB quickly if you already have the data and metadata data checksums. ZFS (with dedupe on) checks the checksum of every written block against every existing block in the pool, thus the slow writes and being very ram intensive. Btrfs instead calculates data and metadata checksums for integrity then allows using them afterwards for deduplication. You could of course check for block level checksums for 100% of your storage. Or you could decide to dedupe only old files, only VMs, or only non-VMs. Or even a hybrid of block level for VMs, but file level for non-VMs.