3 ms·

Does Internet Archive have a distributed residential IP crawler program? I would enthusiastically contribute to that. There must be some mechanism to prevent t

by xnx 8mo ago

Does Internet Archive have a distributed residential IP crawler program? I would enthusiastically contribute to that. There must be some mechanism to prevent tampering in such a setup.

progval 8mo ago
The Internet Archive does not, but Archive Team does: https://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior https://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior
- xnx 8mo ago
  Yes! I'm running an instance right now.
Retr0id 8mo ago
> There must be some mechanism to prevent tampering in such a setup. Trivial as long as they terminate the TLS on their end, not yours. So you'd just be a residential proxy.
gzread 8mo ago
No, IA does everything above board and even honors invalid DMCA takedowns.