3 ms·
nice write up. interesting idea too... very cool... one question though: I wonder what ZFS + DeDup or Windows Storage Spaces + DeDup would do with the same data
by tiernano 7y ago
nice write up. interesting idea too... very cool... one question though: I wonder what ZFS + DeDup or Windows Storage Spaces + DeDup would do with the same data, just out of curiosity...
- binwiederhier 7y agoThe idea here is to do this on a very very very large scale. My employer has a > 500 PB cloud of (mainly Windows) backups. Right now everything is on ZFS, but we're thinking about moving to a block store (like Ceph or Swift) with this dedup idea.
- binwiederhier 7y agoThat said, block level dedup on NTFS will lead to better dedup ratios, but at the cost of a lot more metadata and a very high metadata to data ratio.
- tiernano 7y agoThat makes more sense then. Plus, the larger the data set, the more chance (in theory) of duplicates... especially windows system files, etc.