3 ms·
Interesting, and it sucks because this kind of feels like selective application of a law. Google probably violates this all the time with automated crawling (I
by new299 10y ago
Interesting, and it sucks because this kind of feels like selective application of a law.
Google probably violates this all the time with automated crawling (I've no doubt that there are sites without robots.txt, but with a written ToS that prohibits use of "automated scripts".