3 ms·

Pardon my ignorance, but I'm still not quite understanding the applications and benefits. Distributed file hosting? High-concurrency reads? Torrent-like? "file

Pardon my ignorance, but I'm still not quite understanding the applications and benefits. Distributed file hosting? High-concurrency reads? Torrent-like? "file sharing network" is a bit ambiguous to me. (I seem to have broke the demo with a 100 MB file, btw.)

mindslight 11y ago
(Not my project. I'm just commenting on the general techniques) Rabin chunking should get you quick cross-file partial-file deduplication. So if you're already storing a file and then download a modified version (from someone else) with changed metadata in the header, only the differing data should be transferred/stored.