3 ms·
On smaller sites, nothing usually. But on bigger sites you will be blocked. You will probably be blocked even if you do follow robots.txt
by foobar33333 6y ago
On smaller sites, nothing usually. But on bigger sites you will be blocked. You will probably be blocked even if you do follow robots.txt