3 ms·
In principle? Of course. I mean I remember blocking Yandex bots from hammering some e-commerce site on shoestring budget. But each person developing a web scra
by 3LazTjBv-f 3y ago
In principle? Of course. I mean I remember blocking Yandex bots from hammering some e-commerce site on shoestring budget.
But each person developing a web scrapping bot realizes at most after a week that being honest with User-agent has negative impact on how well it works, and changing it to existing browser takes literally seconds.