4 ms·
Volunteers run Docker containers or Virtualbox VMs at home so that the traffic looks residential and does not get banned. For example: https://imgur.com/a/QXrhu
by smarx007 5y ago
Volunteers run Docker containers or Virtualbox VMs at home so that the traffic looks residential and does not get banned. For example: https://imgur.com/a/QXrhudA https://imgur.com/a/QXrhudA
Most useful content gets packaged by the ArchiveTeam and send to the Internet Archive (no affiliation between the two).
- bshipp 5y agobut it's not really necessary for Reddit. their API is fairly robust and there are numerous options for scraping the site.
- smarx007 5y agoAT is not limited to Reddit scraping, take a look at their wiki.
- bshipp 5y agoI'm aware of AT and the work they do. Its a great initiative. I'm just saying that other projects like https://pushshift.io/ https://pushshift.io/ have been capturing all reddit posts and comments for years now.
- ruined 5y agoreddit will ipo soon, which is why this is frontpaged now, i presume. folks expect the api to soon become much more restrictive. other mirror projects don't have the ideology or reputation of archiveteam.