4 ms·
IA is fine. Archiveteam, which my comment references, has been doing it against user wishes. https://www.archiveteam.org/index.php?title=Mastodon https://www.
by sascha_sl 7y ago
IA is fine.
Archiveteam, which my comment references, has been doing it against user wishes.
https://www.archiveteam.org/index.php?title=Mastodon https://www.archiveteam.org/index.php?title=Mastodon
https://www.archiveteam.org/index.php?title=robots.txt https://www.archiveteam.org/index.php?title=robots.txt
They're not the same entity, although people often assume they are.
- dredmorbius 7y agoI'm aware of Archiveteam, and worked with them on the G+ archive project. They've been sensitive to exclusion requests, and serve principally to move data to IA, where, again, removal requests are easy and recognised. IA have generally the same policy as AT on robots.txt, for entirely justifiable reasons. In the grand scheme of things, open, public, accessible archives >>> closed, private, nonaccessible (or accessible-only-for-a-(price|class) archives. Adding the capacity to denote specific exclusions assists further in this.