5 ms·
So if you find something you'd better make a copy of it yourself because it might be going dark. Doesn't that kind of defeat the whole purpose of the Wayback M
by rmason 9y ago
So if you find something you'd better make a copy of it yourself because it might be going dark. Doesn't that kind of defeat the whole purpose of the Wayback Machine?
- SllX 9y agoYes, but it's also always been true that if you want to keep something for reference, you make your own copy.
- Buge 9y agoThe problem with that though is that people will think you've manipulated your copy. I've had people accuse me of this when I save pages with screenshots. You need to have a trusted third party make and store the copies. Sometimes I use archive.is , they don't automatically delete because of robots.txt but it's not fully clear to me when they do delete things.
- foota 9y agoThis would be entirely overkill, but in theory if you're accessing the site via https couldn't you record the conversation from your end and later prove it?
- rocqua 9y agoYou'd need a proven time stamp on that conversation, or else the site could just switch certs and then leak their keys. Then, they can claim that you forged the traffic using the leaked keys.
- simias 9y agoI can't believe I'm the one to propose this but being able to unjustifiably timestamp some data is one of the only actual use cases of blockchains. Archive your data, compute a checksum and store that in a Bitcoin block and you can prove later on that you actually owned the data at this point. Of course there are other ways to achieve that such as publishing your checksum to a vast number of neutral third parties such as in a mailing list, bittorent or even a newspaper. You could also rely on a trusted 3rd party who has low incentives to manipulate the data (or would risk a lot if they were caught cheating) such as a bank, insurance or notary for instance. I think archive.org could potentially do something like that by using a merkle tree containing the checksums of the various pages they've archived during the day and publish the top hash every day for anybody to archive (or publish it on a blockchain or whatever, as said above). If later on somebody accuses the archive of manipulation they can publish the merkle tree of the day the site was archived which contains the checksum of the page, and anybody having the top hash of that day can vouch that it is indeed correct. It doesn't stop the archive from storing bogus data but it makes it impossible to "change the past" post-facto, so in this particular situation the journalist could only complain that the archive stored bogus data back in the day and not that it was recently tampered with.
- dumbfounder 9y agoI immediately had the same idea to use a 3rd part to host checksums, surprised they haven't done this. Blockchain makes a lot of sense from the immutability standpoint, but how would you incentivize people to maintain it? Maybe you can get people to do that for the common good a la wikipedia? Not sure about that. Maybe you get Apache to bake it into their webserver to ask people to opt-in to dedicate 0.1% of resources to the cause?
- simias 9y agoI was thinking about using an existing blockchain such as Bitcoin. Of course then the inconvenient is that archive.org would have to pay a fee every time they submit a new hash. A comment above pointed out that the scheme I described (unsurprisingly) already exists at https://petertodd.org/2016/opentimestamps-announcement https://petertodd.org/2016/opentimestamps-announcement Realistically it might be overkill though, simply setting up some mailing list where anybody can subscribe and be sent the checksum every day or even just publishing it at some URL and letting users scrap it if they want might be sufficient. If we're talking about one checksum every day it's only a few kilobytes every year, it shouldn't be too difficult to convince a few hundred people and organizations around the world to mirror it.
- dumbfounder 9y agoI think you need a checksum for every page, not every day. How would you independently verify the checksum for an entire day?
- rocqua 9y agoMerkle trees. Whomever wants to store a timestamp for some message stores the path from that message to the root of the Merkle tree. Only the root of the merkle tree of each day needs to be published.
- foota 9y agoEdit: thought about this some more, I don't think this would work since in ssl iirc you agree on a symmetric encryption key which is then used to encrypt the rest of the request response cycle.
- Buge 9y agoNo. HTTPS doesn't provide non-repudiation. HTTPS does not sign the content. It MACs the content.
- 3pt14159 9y agoYup! This is the problem. There’s a blog post out there by a security guy tittled “nothing is real” where he covers how nobody can truly trust the data they get from computers.
- casefields 9y agoDo you think you can find it? Sounds interesting.
- 3pt14159 9y agoI found my tweet about it: https://mobile.twitter.com/zachaysan/status/959485521843097600 https://mobile.twitter.com/zachaysan/status/9594855218430976... As an aside, I’ve been noticing Google has been getting worse and worse at finding something I’m sure is out there. I’m not saying that they’re necessarily getting worse, but maybe it’s getting harder to deal with the sheer scale of the web these days.
- zerebubuth 9y agoOne method for making a copy or crawl very difficult to tamper with is to publish a hash somewhere difficult to forge (e.g: in a national newspaper or opentimestamps). That won't prove the copy wasn't manipulated before it was archived, though. For that, we would need multiple, independent archives. This is effectively what libraries have been doing for many years with their archives of newspapers.
- prepend 9y agoThe hashing needs to be done by a trusted third party. It would be a cheaper to operate service than wayback, but would let you check individuals content against manipulation. You have to incentivize people running and storing the hashes.
- Buge 9y agoYou could put it in the bitcoin blockchain. Or if you don't need that level of complexity and cost, you could put it on twitter, which doesn't allow editing tweets (but does allow deleting).
- SllX 9y agoIt's good enough to combat link rot if you need to or want to refer back to or read something again. Nothing on the internet is permanent and even if it is still out there, that's no guarantee that Google or any other search engine still has it in their index. At least that way the information isn't lost to you.
- lzy 9y agoSo I try to make a copy of any interesting web pages on archive.is these days. http://archive.is/faq#Why_does_archive_is_not_obey_robots_txt_ http://archive.is/faq#Why_does_archive_is_not_obey_robots_tx...
- nsbq71 9y agoThis one is funny, because conservatives have used archive.is for some time to archive and mock left-leaning websites and some of them blocked archive.is in the past and still block archive.is today. VOX for example returns a 0-sized page for archive.is. In the past VICE returned 404s to archive.is https://i.imgur.com/OnFdVpS.jpg https://i.imgur.com/OnFdVpS.jpg What I mean to say is that these services are useful but they are not faultless.
- SmellyGeekBoy 9y agoWhy are so many irrelevant political left-vs-right "he said, she said" type comments popping up on HN just lately?
- fapjacks 9y agoI personally think this is simply a result of how much harder the media is pushing that divide (for all of their various purposes). I actually spent some time last year researching this, because I thought I might have just become an old man thinking how great things used to be. I started reading old news stories fairly randomly, from the present time all the way back to the Vietnam era (and a few rabbit holes to earlier times). The first thing that surprised me was the amount of link rot that exists. I always knew intellectually that it was a problem, but wow. It's bad. The second thing that I found was that indeed, the media hammers on the "us-versus-them" political divide of American politics much, much harder nowadays than even ten years ago. I think Fox News was really the turning point. It opened the flood gates. I always remember thinking how "extreme" Fox News was, but I challenge anyone to look up a few of their older stories from the middle of the last decade. It's child's play compared to what pretty much every media outlet is doing today. You can hardly read a recent news story from just about anywhere without being told how it's supposed to fit into our political worldview, and how we should feel about it, and why it's good/bad/stupid/amazing/"terrifying". And so of course, because of this, people are just responding to the programming. Creating the world they're led to believe we they live in. I think it really is that straightforward.