3 ms·
Courtesy is a practical way of avoiding legal disputes. If you ignore robots.txt, that means you don’t have an agreement, so the question is what do you have a
by skybrian 2y ago
Courtesy is a practical way of avoiding legal disputes. If you ignore robots.txt, that means you don’t have an agreement, so the question is what do you have a right to do, and do you really want to argue about that?
- lagniappe 2y agoYou're free to bring anything to court, that's your right.