4 ms·
That was... vague Why would the robots file on an active site be applied to the archived content?
by ry_ry 8y ago
That was... vague
Why would the robots file on an active site be applied to the archived content?
- paulie_a 8y agoI agree. That sort of defeats the purpose of something that is literally meant to archive
- ry_ry 8y agoTeMPOraL had the most plausible explanation up thread a bit - apparently it's there as a copyright claim safety-net for Wayback Machine, but making a feature that runs counter to the whole point of the site publicly available makes my head hurt. Perhaps having a bot generate a synopsis of removed content, and showing that in its place would solve any copyright issue fairly elegantly?
- icebraining 8y agoGenerating a synopsis might have other implications (e.g. accusations of libel if the original author considers the synopsis attributes false claims to them).
- occamrazor 8y agoA synopsis is a derivative work. As such, it falls within copyright laws.
- ry_ry 8y agoHuh,I'd always thought they fell under fair use in the same way a review does. You learn something new every day... Multiple times a day in my case!
- pygy_ 8y agoAgreed, it doesn't make any sense to apply those rules retroactively.
- zxcmx 8y agoLegal issues. Who owns the content? There is no real legal basis for a right to mirror, despite how it feels from a techy point of view.
- amelius 8y agoRobots.txt now has legal status?
- SiempreViernes 8y agoIt undoubtedly communicates some intention by the site owner about how the work on the site can be used, so probably?
- amelius 8y agoIt only communicates the intention if the reader is actually obliged to read and interpret the file as a legal document. If I put a file named "legal.txt" in an online folder, is anyone required to read it and act upon it? It might as well be a file intended for some completely unrelated purpose; e.g. a lawyer that put some drafts online, or for all the reader knows, it might even be part of a movie script.
- pseudalopex 8y agorobots.txt has been a de facto standard for over 20 years. Someone might be able to claim ignorance, but the Internet Archive has shown that they know about it. It has a specific format; if it can be parsed, it's safe to assume that it isn't part of a movie script. In most cases, copyright law requires the reader of a document not to republish it, so the robots.txt standard is actually much more permissive.
- occamrazor 8y agoIt represents the expressed preferences of the website owner, so it is legally relevant.
- 8y ago
- skywhopper 8y agoPragmatic decision to avoid big stinks that might risk the extremely high ratio of sites that _don't_ currently block IA.