3 ms·
No, but in whatever little bot/scraper blocking work I did, user-agent blocking was an incredibly powerful low-hanging fruit. For sensitive material, we would
by nulld3v 2y ago
No, but in whatever little bot/scraper blocking work I did, user-agent blocking was an incredibly powerful low-hanging fruit.
For sensitive material, we would often insta-ban anyone who hit us with a 'curl' user-agent and that definitely killed a lot of script-kiddies' dreams right then and there.
Obviously, if you are a determined "hacker" you would get past that and hit the honeypots but simple filters could filter out enough scrapers such that the rest of the attacks could be manually evaluated by humans.
- LtWorf 2y agoIt feels very strange to me that someone can honestly believe that google isn't pulling shenanigans.