4 ms·
You don't have to request anything. Just alter robots.txt and you can make the Wayback Machine memory hole your entire website.
by J_Darnley 10y ago
You don't have to request anything. Just alter robots.txt and you can make the Wayback Machine memory hole your entire website.
- idsout 10y agoThe keyword here is 'your'.
- sp332 10y agoWho else would be sending takedown requests?
- Skunkleton 10y agoContent owners that already DCMA'd their content on some third parties website which the wayback machine has backed up.
- dingaling 10y agoOr someone else's website to which you now own the domain name
- minikites 10y agoIt's so infuriating when I come across a domain squatter that nuked the entire history of a domain in the Wayback Machine. I sort of get why they have to do that but it also defeats most of the point of the Wayback Machine.
- slrz 10y agoI sort of get why they have to do that I don't. Can you explain?
- btgeekboy 10y agoArchive.org doesn't know the domain changed hands, just that it used to be allowed to show the results but now no longer is.
- dragonwriter 10y agoDoesn't really explain why they have to nuke it, even if it is the current site owner. Respecting robots.txt is one thing, but that just means not spidering and archiving the content that is now there. Deleting already archived material based on later changes to robots.txt is a non-obvious behavior, given the usual understanding of the general meaning of robots.txt.
- aab0 10y agoThey're not deleting it, just hiding it from public access. Once the squatter goes away, the content comes back.
- J_Darnley 10y agoWhat's the difference? Both make this feature (and more general use of the archive) useless.
- aab0 10y agoThe difference is exactly what I said: if they deleted it, it's gone forever. If they hide it, it can come back. I've seen pages I cite disappear for a year or two thanks to scummy squatters - but they came back! It's the difference between being sentenced to execution and to 1 year of prison.
- infogulch 10y agoCould they not also watch when a domain changes ownership and segment history based on the owner? Over the time scales that archive.org holds on to data, domain ownership itself becomes part of the history. While permitting someone to hide a mistake for security reasons is reasonable, allowing erasure of past owners' history by the current owner is counter to their stated purpose.
- Dylan16807 10y agoHave you tried contacting them about such domains?