5 ms·

As someone who uses the search function on HN regularly I have to agree its pretty jarring to see how many webpages totally disappear. Thankfully the wayback ma

As someone who uses the search function on HN regularly I have to agree its pretty jarring to see how many webpages totally disappear. Thankfully the wayback machine exists.

yen223 5y ago
One day, the wayback machine will start returning 404s too
- AnimalMuppet 5y ago
  That's... a bit sobering. At that point, we will really have lost a lot of history.
  - twodai 5y ago
    It's possible some other group will buy the data, or they will make it easily download able
    - jlokier 5y ago
      Like the way Google did with the Usenet archive. Make it available via a searchable interface, then gradually degrade the archive until old posts can't be found any more.
      - squarefoot 5y ago
        They also degraded the interface so that the posts that once would be found via normal searching now aren't anymore. After the removal of the discussion filter, searches for X started returning more sites selling X than forums talking about X. https://www.reddit.com/r/google/comments/2b54ux/google_completely_removed_discussions_search_is/ https://www.reddit.com/r/google/comments/2b54ux/google_compl... There was a thread on Google forums about that, and I recall many upset users asking for the option to be reactivated, but (the irony) that discussion was removed as well.
    - metagame 5y ago
      Actually, archive.org is already easily downloadable, in that you can already help host the dweb copy. archive.org, available via webtorrent: https://dweb.archive.org/ https://dweb.archive.org/ It's pretty slow, though.
- dotancohen 5y ago
  I think that the Wayback machine supports either an HTTP status code or meta tag that causes it to not serve previously-cached contents. Therefore I try to save webpages that I care about, but it's getting harder and harder. Not to mention the space it takes - is it really worth the hundreds of GiB of personal archives when I'll likely want - not need - maybe a few KiBs of it decades down the line. And even then, will I be able to find it? Sometimes I think about just archiving a screenshot and the text of websites, instead of markup and related files.
- dredmorbius 5y ago
  One day there will be no more 404s or 200s.
omn1 5y ago
I maintain a couple of bigger Github repos and blogs and was shocked at how many links break on a regular basis, so I wrote my own link checker in Rust [1] and started donating to the Wayback Machine. Please consider doing so, too as the web would be a worse place without them. [2] [1]: https://github.com/lycheeverse/lychee https://github.com/lycheeverse/lychee [2]: https://archive.org/donate/ https://archive.org/donate/

As someone who uses the search function on HN regularly I have to agree its pretty jarring to see how many webpages totally disappear. Thankfully the wayback machine exists.

yen223 5y ago

One day, the wayback machine will start returning 404s too

AnimalMuppet 5y ago
That's... a bit sobering. At that point, we will really have lost a lot of history.
- twodai 5y ago
  It's possible some other group will buy the data, or they will make it easily download able
  - jlokier 5y ago
    Like the way Google did with the Usenet archive. Make it available via a searchable interface, then gradually degrade the archive until old posts can't be found any more.
    - squarefoot 5y ago
      They also degraded the interface so that the posts that once would be found via normal searching now aren't anymore. After the removal of the discussion filter, searches for X started returning more sites selling X than forums talking about X. https://www.reddit.com/r/google/comments/2b54ux/google_completely_removed_discussions_search_is/ https://www.reddit.com/r/google/comments/2b54ux/google_compl... There was a thread on Google forums about that, and I recall many upset users asking for the option to be reactivated, but (the irony) that discussion was removed as well.
  - metagame 5y ago
    Actually, archive.org is already easily downloadable, in that you can already help host the dweb copy. archive.org, available via webtorrent: https://dweb.archive.org/ https://dweb.archive.org/ It's pretty slow, though.
dotancohen 5y ago
I think that the Wayback machine supports either an HTTP status code or meta tag that causes it to not serve previously-cached contents. Therefore I try to save webpages that I care about, but it's getting harder and harder. Not to mention the space it takes - is it really worth the hundreds of GiB of personal archives when I'll likely want - not need - maybe a few KiBs of it decades down the line. And even then, will I be able to find it? Sometimes I think about just archiving a screenshot and the text of websites, instead of markup and related files.
dredmorbius 5y ago
One day there will be no more 404s or 200s.

omn1 5y ago

I maintain a couple of bigger Github repos and blogs and was shocked at how many links break on a regular basis, so I wrote my own link checker in Rust [1] and started donating to the Wayback Machine. Please consider doing so, too as the web would be a worse place without them. [2] [1]: https://github.com/lycheeverse/lychee https://github.com/lycheeverse/lychee [2]: https://archive.org/donate/ https://archive.org/donate/