3 ms·

If they really want to be trusted as a source of truth, they need to find a way to publish cryptographic signatures of every page that can be validated to prove

If they really want to be trusted as a source of truth, they need to find a way to publish cryptographic signatures of every page that can be validated to prove that the page was captured when they say it was. Maybe some sort of blockchain. Otherwise there's no reason to trust that an Internet Archive snapshot is an accurate representation of the archived page.

seppin 7y ago
> Maybe some sort of blockchain. Or you know, any other kind of secure database.
- Johnny555 7y ago
  Just curious, what kind of secure database lets anyone independently verify the integrity without needing to trust the entity that hosts the database? I can envision a chain of signatures that lets anyone validate a complete chain of website versions, but am not familiar with a database that allows the same.
philips 7y ago
I think a transparency log of all URLs that they put into the archive would be a good step. This at least guarantees that a URL isn't modified after getting into the archive. Some prior art: https://blog.golang.org/module-mirror-launch https://blog.golang.org/module-mirror-launch https://www.certificate-transparency.org/what-is-ct https://www.certificate-transparency.org/what-is-ct https://merklecounty.substack.com/p/rget-a-secure-download-user-story https://merklecounty.substack.com/p/rget-a-secure-download-u...
dredmorbius 7y ago
I think that's included in the WARC format. OK: it's an optional field, "checksum", but not a formal has. As of v. 0.9: http://archive-access.sourceforge.net/warc/warc_file_format-0.9.html http://archive-access.sourceforge.net/warc/warc_file_format-... I haven't checked the drafts specs for further progress, but that's here: http://archive-access.sourceforge.net/warc/ http://archive-access.sourceforge.net/warc/ IA are instrumental in developing WARC, use it as their archival format (non-WARC web archives are ... generally not acceptable), and it's been adopted by US Library of Congress and other institutions as a standard.
khawkins 7y ago
I wouldn't say "no reason", I'd just say that it should be self-aware that they are functionally similar to a news website in that they live and die on their reputation. The second someone can demonstrate that they've manually misrepresented some archived page they lose nearly all of that credibility. Better than crypto stuff which might not scale well would be to have multiple independent archives run by people with a diversity of political views and nationalities. The authenticity can be easily verified if the same page is archived to a site run by left-wing people in San Francisco and also a site run by right-wing people in London.
- Johnny555 7y ago
  But you need more than two, or who would break the tie when the left-leaning site says one thing, and the right-leaning site says the opposite. Running a general internet archive isn't cheap, so it's not like there would be dozens of them.
_pmf_ 7y ago
Would storing the SSL/TLS session achieve this? Is this possible using browser means (HAR export)?
commandlinefan 7y ago
That's the opposite of what they're trying to do, though. They're not trying to preserve what was printed, but to get rid of some of those things entirely as if they had never existed.

seppin 7y ago

> Maybe some sort of blockchain. Or you know, any other kind of secure database.

Johnny555 7y ago
Just curious, what kind of secure database lets anyone independently verify the integrity without needing to trust the entity that hosts the database? I can envision a chain of signatures that lets anyone validate a complete chain of website versions, but am not familiar with a database that allows the same.

philips 7y ago

I think a transparency log of all URLs that they put into the archive would be a good step. This at least guarantees that a URL isn't modified after getting into the archive. Some prior art: https://blog.golang.org/module-mirror-launch https://blog.golang.org/module-mirror-launch https://www.certificate-transparency.org/what-is-ct https://www.certificate-transparency.org/what-is-ct https://merklecounty.substack.com/p/rget-a-secure-download-user-story https://merklecounty.substack.com/p/rget-a-secure-download-u...

dredmorbius 7y ago

I think that's included in the WARC format. OK: it's an optional field, "checksum", but not a formal has. As of v. 0.9: http://archive-access.sourceforge.net/warc/warc_file_format-0.9.html http://archive-access.sourceforge.net/warc/warc_file_format-... I haven't checked the drafts specs for further progress, but that's here: http://archive-access.sourceforge.net/warc/ http://archive-access.sourceforge.net/warc/ IA are instrumental in developing WARC, use it as their archival format (non-WARC web archives are ... generally not acceptable), and it's been adopted by US Library of Congress and other institutions as a standard.

khawkins 7y ago

I wouldn't say "no reason", I'd just say that it should be self-aware that they are functionally similar to a news website in that they live and die on their reputation. The second someone can demonstrate that they've manually misrepresented some archived page they lose nearly all of that credibility. Better than crypto stuff which might not scale well would be to have multiple independent archives run by people with a diversity of political views and nationalities. The authenticity can be easily verified if the same page is archived to a site run by left-wing people in San Francisco and also a site run by right-wing people in London.

Johnny555 7y ago
But you need more than two, or who would break the tie when the left-leaning site says one thing, and the right-leaning site says the opposite. Running a general internet archive isn't cheap, so it's not like there would be dozens of them.

_pmf_ 7y ago

Would storing the SSL/TLS session achieve this? Is this possible using browser means (HAR export)?

commandlinefan 7y ago

That's the opposite of what they're trying to do, though. They're not trying to preserve what was printed, but to get rid of some of those things entirely as if they had never existed.