2 ms·
That's a great explanation, thank you. How does this handle versioning? If I add one file to a vast dataset, can people download just that file if they already
by StavrosK 7y ago
That's a great explanation, thank you. How does this handle versioning? If I add one file to a vast dataset, can people download just that file if they already have the previous version? IPFS hashes at the chunk level, so even if you append something to a large file, you can download just the new chunk and be up to date.
- rakoo 7y agoYes, in fact dat was initially created for this use case: data analysts want to exchange their data, and that data evolves in time so it needs to be transported "efficiently", as in, you don't need to redownload a full .zip just for a single file change. The only metadata you'll receive will concern the new file and the old content is still valid. You can even seek inside any file if you don't want to download the whole file. Changes _inside_ a file, though, are not handled. Today if a file is modified dat will consider any bytes of the old one to be garbage and will not reuse it. dat is a sexy frontend on top of hyperdrive(https://github.com/mafintosh/hyperdrive https://github.com/mafintosh/hyperdrive), I personally think it's easier to see what dat can do by looking at what hyperdrive does
- StavrosK 7y agoThanks, that's very informative, and the intro in the Hyperdrive README clarifies the goals very well. I have a much better idea now, thanks again.