3 ms·
I'd like to see an actual case reference for this. I've never heard of ignoring robots.txt resulting in any kind of legal action. It would take a pretty big le
by c2 16y ago
I'd like to see an actual case reference for this. I've never heard of ignoring robots.txt resulting in any kind of legal action.
It would take a pretty big leap to go from robots.txt is advisory to ignoring it constitutes a criminal action.
- jasonwatkinspdx 16y agoInternet Archive was sued unsuccessfully. As I understand it a lawsuit is still in process against Google on the topic. So I guess the precedent is weaker than I thought, but still: tread carefully.