2 ms·
I think that is not what they mean. I think what they meant is the site will detect your headless robot and serve it good content, while serving spam and malwar
by jeffbee 4y ago
I think that is not what they mean. I think what they meant is the site will detect your headless robot and serve it good content, while serving spam and malware to everyone else. The crawlers need their own distributed networks of unrelated addresses to prevent or detect this behavior.