4 ms·
A robots.txt file doesn’t block scrapers, it’s the equivalent of a no trespassing sign. I don’t think putting up signs will suddenly become illegal?
by herval 7y ago
A robots.txt file doesn’t block scrapers, it’s the equivalent of a no trespassing sign. I don’t think putting up signs will suddenly become illegal?