4 ms·

For a dynamic solution massive redundancy and some kind of stakeholder accounting. Some kind of "planetary file system" established in the public interest. Are

For a dynamic solution massive redundancy and some kind of stakeholder accounting. Some kind of "planetary file system" established in the public interest. Are the Wikipedia and Internet Archive good models? Technically a kind of thing like a benevolent version of Web3.0 might solve if it were not motivated solely by profit. IPFS and variants are the landscape at the moment. The consideration (reward) for participation is access to the whole corpus. You need to solve abuse problems which mean Wikipedian-like entry guards and curating of content insertion. Do you trust them? We'd need at least 10x redundancy, so imagine more like a few million people with 10TB to spare. That's not unreasonable in the next few years. But that needs to be maintained or else swathes of data will be lost forever if not enough copies are kept. That means you need at least a few million people who seriously give a shit, and those are getting harder to find each day. Also, with advances in compression, that could be a lot less space if you're prepared to trade-off access time. Like going to the library to find and scan a newspaper, image making a request that takes the distributed system several minutes or hours to lookup, assemble, collect and decode all the pieces. That doesn't sit well with a "give it me now!" culture. For static a solution, burying a few petabytes of long-range storage at strategic (maybe secret) locations is an assurance against Nineteen Eighty Four/Fahrenheit 451 scenarios - a library to outlive the next tin-pot "Thousand year Reich". But that doesn't give anyone quick/random access to disputed facts and records. A more political/human solution is to get more people caring about history and truth, ad-hoc curation and preservation. That requites not just liberation of data a la Aaron Swartz and Alexandra Elbakyan, but hugely increasing the number of people who will participate in that project. At this point, belief in the preservation of historical human knowledge means fighting the law,

kaurov 3y ago
Neither wikipedia or Internet Archive are fully protected from someone tampering their databases. And while I want to believe that they have security sorted out, it demands me to blindly trust them. I like and use Wikipedia but I do not trust it 100% (not because of possible tampering but simply because of the way it is assembled). Giving access to the corpus as a reward is great but the problem is that very few people will actually want to use it :). On the other hand slow access time should not be the problem for those few who care. Maybe the best combination is to continue using fast access systems (like existing digital libraries) and then slow systems only for confirmation. Curation wiki-style might not even be the main issue at the beginning because a lot of things are already pre-curated. Like old radio recordings or newspaper archives. Curation will become a nightmare with modern media like twits… Changing people’s minds would be great, but I am afraid AI is evolving on a much shorter timescale. :)
- nonrandomstring 3y ago
  Sorry for the late reply. Yea its an interesting problem space and one that belongs squarely in the public interest area. Public interest and commons problems look hard because the incentives look hard. They are not always if you use the right craft and leverage, The GPL is a masterpiece of guile when you think about it. Another way to build trust is legally by creating some very tightly drafted and constantly lawyer-policed consortium that gets donations from bigger players but ensures selfish interests are never allowed to prosper. Such a network would need to have the power to literally tell Google and Apple to fuck off without seriously impacting itself, and that kind of distributed power/spreading would be hard to maintain against constant attempts at capture. By the way, look at what the BBC have been doing with content provenance and media supply chain assurance. Its possible some big owners will watermark and release new commons in the future in such a way as to ensure it's tamper-proof.
  - kaurov 3y ago
    Thank you for your input! It is interesting to imagine what consortium of data hosting organization could provide the ultimate trust platform. Some people actually do trust big tech, others trust government or churches, most people in the US trust the military. Therefore, maybe it is possible to imagine a feasible alliance of them that could manage hosting a huge archive.

kaurov 3y ago

Neither wikipedia or Internet Archive are fully protected from someone tampering their databases. And while I want to believe that they have security sorted out, it demands me to blindly trust them. I like and use Wikipedia but I do not trust it 100% (not because of possible tampering but simply because of the way it is assembled). Giving access to the corpus as a reward is great but the problem is that very few people will actually want to use it :). On the other hand slow access time should not be the problem for those few who care. Maybe the best combination is to continue using fast access systems (like existing digital libraries) and then slow systems only for confirmation. Curation wiki-style might not even be the main issue at the beginning because a lot of things are already pre-curated. Like old radio recordings or newspaper archives. Curation will become a nightmare with modern media like twits… Changing people’s minds would be great, but I am afraid AI is evolving on a much shorter timescale. :)

nonrandomstring 3y ago
Sorry for the late reply. Yea its an interesting problem space and one that belongs squarely in the public interest area. Public interest and commons problems look hard because the incentives look hard. They are not always if you use the right craft and leverage, The GPL is a masterpiece of guile when you think about it. Another way to build trust is legally by creating some very tightly drafted and constantly lawyer-policed consortium that gets donations from bigger players but ensures selfish interests are never allowed to prosper. Such a network would need to have the power to literally tell Google and Apple to fuck off without seriously impacting itself, and that kind of distributed power/spreading would be hard to maintain against constant attempts at capture. By the way, look at what the BBC have been doing with content provenance and media supply chain assurance. Its possible some big owners will watermark and release new commons in the future in such a way as to ensure it's tamper-proof.
- kaurov 3y ago
  Thank you for your input! It is interesting to imagine what consortium of data hosting organization could provide the ultimate trust platform. Some people actually do trust big tech, others trust government or churches, most people in the US trust the military. Therefore, maybe it is possible to imagine a feasible alliance of them that could manage hosting a huge archive.