3 ms·
Thanks for actually looking into the mentioned problems instead of going on a tangent about climate change ;) CacheP2P seems to solve the problem by including
by oceanofsolaris 10y ago
Thanks for actually looking into the mentioned problems instead of going on a tangent about climate change ;)
CacheP2P seems to solve the problem by including a list of sha1 hashes for all files that are distributed through P2P.
Speculation time:
I guess distributing the sha1 hashes for every wikipedia page is a bit too much (especially considering they are always changing). So with every page, you would only distribute the hash for those sites that are directly linked from it.
If you provide the correct sha1 hash for every linked website (which in turn includes the hashes of all linked websites from it...), you would form some kind of merkle-tree of wikipedia pages. Except that the graph of wikipedia pages is certainly highly cyclic (thus no tree), so this isn't feasible. And all the pages change all the time. So this naive approach has already two strikes against it and does certainly not work.
I will try to have a look at the source code at some later point, I guess.
- posterboy 10y agoEdit: 2.5 Mb was wrong, shame on me -_- 5 Million articles, 512b per Hash, if that's enough, gives 2.5 Gb. Updates can be much smaller. Multiply that by 10 for hashes of the nearest users, who have a copy, and that's still manageable. Just of the top of my head. edit: inb4 a blockchained approach.
- gcr 10y agoIt's 320 MB. 5 million articles times 64 bytes per hash (512 bits) divided by 1e6 bytes per MB is 320.
- oceanofsolaris 10y agoOk, that's not too unreasonable (especially if you need to download an extension anyways). If you make delta updates available for changed pages, this should actually be reasonably small once you have done the initial download.