4 ms·
Hmm. Is having 'robots.txt' a requirement. What if one just changes the terms and conditions page? Can the bot figure out it just committed a crime?
by morekozhambu 10y ago
Hmm. Is having 'robots.txt' a requirement. What if one just changes the terms and conditions page? Can the bot figure out it just committed a crime?
- chii 10y agoIf you yelled at a deaf person to go away, are you surprised that they don't comply?
- gm-conspiracy 10y agoIf I am facing them, and they can read lips, then yes.
- erroneousfunk 10y agorobots.txt has no legal standing by itself, but is a convenient way to translate "terms of service" (which do have legal meanings, but they're often more limited than people assume) into something that bots can automatically understand. You're under no legal obligation to follow robots.txt, but it's a tool to prevent stepping on toes.
- Animats 10y agoThe "robots.txt" file has figured in two copyright cases.[1] Courts have taken notice of it as an indication of the web site operators's intent. [1] http://www.law360.com/articles/497068/the-role-of-the-robot-exclusion-in-copyright-defenses http://www.law360.com/articles/497068/the-role-of-the-robot-...