Dedupe is a lot less memory-intensive if you have larger files and tune your datasets' `recordsize` to 1MiB instead of the 128KiB default. It makes per-file ded

Dedupe is a lot less memory-intensive if you have larger files and tune your datasets' `recordsize` to 1MiB instead of the 128KiB default. It makes per-file deduplication less efficient but makes the overall process much more usable since there will be much fewer total blocks for it to keep track of. I use it to maintain my torrent seed directories while also moving/retagging the files into my own separate hierarchy.