9 ms·
Robots.txt doesn't confer copyright. What about domains that have been sharked ? Does controlling robots.txt now give me the right to suppress all content ever
by sitkack 10y ago
Robots.txt doesn't confer copyright.
What about domains that have been sharked ? Does controlling robots.txt now give me the right to suppress all content ever originating from that domain, for as long as I control robots.txt?
Internet archive is right to spider the site, but defer showing. Collection != dissemenation.
The IA isn't synthesizing, selling, cross referencing or afaict doing anything nefarious with the data.
You are literally picking on the last org on the internet that needs to get picked on.
- cJ0th 10y ago> You are literally picking on the last org on the internet that needs to get picked on. True but at the same time I do understand those who generally want others to abide by their Robots.txt. INAL, but ideally I would love to see the IA having the right to ignore Robots.txt (however, if some one wants to opt out of the IA they should be given the option) while others shouldn't be allowed to do so.