4 ms·
I think this requires more explanation. Is this some kind of cloud archiving? Are they using ipfs or something like that?
by mnd999 5y ago
I think this requires more explanation. Is this some kind of cloud archiving? Are they using ipfs or something like that?
- timdaub 5y agoI'm also not totally familiar with what's going on but I discovered it on /r/datahoarder and I think it's because redditors are scared that content starts vanishing now that reddit files for IPO. Anyways, here's a further description: http://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior http://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior
- Spooky23 5y agoOf course it will. Reddit is probably one the largest porn sites.
- jcrawfordor 5y agoArchiveTeam usually sends all the data they collect into Internet Archive. The two are independent, unrelated organizations, but IA is pretty open about accepting hoards of this type.
- deleted 5y ago[deleted]
- smarx007 5y agoVolunteers run Docker containers or Virtualbox VMs at home so that the traffic looks residential and does not get banned. For example: https://imgur.com/a/QXrhudA https://imgur.com/a/QXrhudA Most useful content gets packaged by the ArchiveTeam and send to the Internet Archive (no affiliation between the two).
- bshipp 5y agobut it's not really necessary for Reddit. their API is fairly robust and there are numerous options for scraping the site.
- smarx007 5y agoAT is not limited to Reddit scraping, take a look at their wiki.
- bshipp 5y agoI'm aware of AT and the work they do. Its a great initiative. I'm just saying that other projects like https://pushshift.io/ https://pushshift.io/ have been capturing all reddit posts and comments for years now.
- ruined 5y agoreddit will ipo soon, which is why this is frontpaged now, i presume. folks expect the api to soon become much more restrictive. other mirror projects don't have the ideology or reputation of archiveteam.