4 ms·

Its simple, and I'll quote myself - "robots.txt isn't the law".

ColinWright 1y ago
Quoting Cervisia : > robots.txt. This is not the law In Germany, it is the law. § 44b UrhG says (translated): (1) Text and data mining is the automated analysis of one or more digital or digitized works to obtain information, in particular about patterns, trends, and correlations. (2) Reproductions of lawfully accessible works for text and data mining are permitted. These reproductions must be deleted when they are no longer needed for text and data mining. (3) Uses pursuant to paragraph 2, sentence 1, are only permitted if the rights holder has not reserved these rights. A reservation of rights for works accessible online is only effective if it is in machine-readable form. -- https://news.ycombinator.com/item?id=45776825 https://news.ycombinator.com/item?id=45776825
- davsti4 11mo ago
  Its the law pertaining to copyright. https://www.gesetze-im-internet.de/englisch_urhg/englisch_urhg.html https://www.gesetze-im-internet.de/englisch_urhg/englisch_ur... ... and that's only in Germany. If what you're protecting with robots.txt isn't copyright-able, then you'll need to find another legal means.
bigbuppo 1y ago
Violating norms makes you an abusive jerk at best.
- davsti4 11mo ago
  That's a slippery slope opinion. One person's norm can oppose another's norm. Who defines them? If my "norm" is to stop at a stop sign, and yours is to roll through it, are you the abusive jerk?
  - bigbuppo 11mo ago
    No means no. Only yes means yes. Stop means stop. Why are people in tech so confused about this? Agree, or ask again in three days.
smsm42 1y ago
"I will do anything that will not literally get me into jail" is a pretty low bar. Most decent people try to do better than that - and that's the only reason society still exists, because there's not enough cops to put all bad people into jail and never will be.
- davsti4 11mo ago
  Exactly - its why laws are written. So, if its enough of a problem, then there should be a law stopping/regulating it. Are published TOSs enough to stop the owners of the bots?
  - smsm42 11mo ago
    TOS wouldn't be enough but laws wouldn't be enough either. We have thousands of pages of laws, and people violate them all the times, and very often with zero consequences.

Quoting Cervisia : > robots.txt. This is not the law In Germany, it is the law. § 44b UrhG says (translated): (1) Text and data mining is the automated analysis of one or more digital or digitized works to obtain information, in particular about patterns, trends, and correlations. (2) Reproductions of lawfully accessible works for text and data mining are permitted. These reproductions must be deleted when they are no longer needed for text and data mining. (3) Uses pursuant to paragraph 2, sentence 1, are only permitted if the rights holder has not reserved these rights. A reservation of rights for works accessible online is only effective if it is in machine-readable form. -- https://news.ycombinator.com/item?id=45776825 https://news.ycombinator.com/item?id=45776825

davsti4 11mo ago
Its the law pertaining to copyright. https://www.gesetze-im-internet.de/englisch_urhg/englisch_urhg.html https://www.gesetze-im-internet.de/englisch_urhg/englisch_ur... ... and that's only in Germany. If what you're protecting with robots.txt isn't copyright-able, then you'll need to find another legal means.

bigbuppo 1y ago

Violating norms makes you an abusive jerk at best.

davsti4 11mo ago
That's a slippery slope opinion. One person's norm can oppose another's norm. Who defines them? If my "norm" is to stop at a stop sign, and yours is to roll through it, are you the abusive jerk?
- bigbuppo 11mo ago
  No means no. Only yes means yes. Stop means stop. Why are people in tech so confused about this? Agree, or ask again in three days.

smsm42 1y ago

"I will do anything that will not literally get me into jail" is a pretty low bar. Most decent people try to do better than that - and that's the only reason society still exists, because there's not enough cops to put all bad people into jail and never will be.

davsti4 11mo ago
Exactly - its why laws are written. So, if its enough of a problem, then there should be a law stopping/regulating it. Are published TOSs enough to stop the owners of the bots?
- smsm42 11mo ago
  TOS wouldn't be enough but laws wouldn't be enough either. We have thousands of pages of laws, and people violate them all the times, and very often with zero consequences.