3 ms·
Interesting, thanks. Not clear to me when reading the announcement is: will hyperdrive replace dat? From reading the comments here, it seems that the answer is
by MayeulC 6y ago
Interesting, thanks. Not clear to me when reading the announcement is: will hyperdrive replace dat? From reading the comments here, it seems that the answer is "yes".
Now, onto the content, you touch on de-duplication. I am quite concerned with the cost associated to updating a large file. Is something like rolling hashes investigated, to chunk files independently of their size? I guess it is, given you seem to be working hard on de-duplication.
But then, that kind of trick best works on uncompressed data, which is inefficient for transmission. Is data compressed before transmission? Whole chunks, or whole files at a time? Ahead of time? Interactively based on what the peer needs?
The trade-offs are many, and complex to investigate. I'm wondering if this could be used as an OS image, like OSTree does?
And lastly, I did not get if multiple peers having the same private key identity could modify the structure simultaneously. What would happen?
Also, nodejs gave me a kneejerk reaction that may be unwarranted, but that's quite a huge dependency to pull in for something that wants to be a ubiquitous building block. Does it have a C API? Also, twitter, discord, github (node to a lesser extent)... it seems somewhat ironic to build a ultimate decentralized filesystem while relying on these hypercentralized offerings, and I am afraid it could turn some contributors off.
- rklaehn 6y agoI kinda share your kneejerk reaction about nodejs for foundational tools, even though I must admit that dat/hyperdrive works surprisingly well for a js tool. But dat/hyperdrive is a well-documented protocol, and there are several ports to other languages. I am very interested in the rust port, but I am not sure in what state it is: https://datrs.yoshuawuyts.com/ https://datrs.yoshuawuyts.com/ I have seen some tweets about progress being made on this. Does anybody know more?
- mafintosh 6y agoThe rust port of Hypercore has been very active recently and they are making good progress. Part of the latest Hypercore release was to move some of the transport crypto to be easier to port to other languages such as rust. The wire protocol works now: https://github.com/Frando/hypercore-protocol-rs https://github.com/Frando/hypercore-protocol-rs and the community is active in #datrs on freenode