3 ms·
Yes. Blocking all crawlers except Google bot is itself a problem. There should not be any bot protection, only abuse (e.g. DDOS) protection. Block disruptive
by account42 3y ago
Yes.
Blocking all crawlers except Google bot is itself a problem.
There should not be any bot protection, only abuse (e.g. DDOS) protection. Block disruptive behaviors, not fingerprints.