5 ms·
Yes, but much of the Wayback Machine’s reddit content was specifically targeted and scraped by ArchiveTeam, who are volunteers that seek out at-risk content fro
by Asparagirl 9y ago
Yes, but much of the Wayback Machine’s reddit content was specifically targeted and scraped by ArchiveTeam, who are volunteers that seek out at-risk content from the web and make sure that it gets into the Wayback. In the past few years we’ve specifically tried to go after sub-reddits that we thought were newsworthy and/or at high risk for deletion. But there’s no way we can get all of it.
But you can help! If you have extra server space/bandwidth or you can spare $40/month, we can add more pipelines:
https://www.archiveteam.org/index.php/ArchiveBot https://www.archiveteam.org/index.php/ArchiveBot
Source: am ArchiveTeam member, run various pipelines, have scraped sub-reddits ranging from The_Donald to the cryptocurrency worlds to darknet markets.