4 ms·

What is your way of detecting them? Just cat your way through your logs?

• sgc 1mo ago
  fail2ban
• cpburns2009 1mo ago
  Almost lol: grep, sort and uniq. If I notice someone is hammering my employer's ecommerce site, I'll block them. It isn't required often so I've been reluctant to spend the time setting up fail2ban.
  • tommica 1mo ago
    Is it a multi-server setup? If so, do you ssh into each machine and look at the logs?
    • cpburns2009 1mo ago
      It's 3 servers so it's not too much hassle to ssh into them and check it manually.
• what 1mo ago
  Google (and other “legitimate” scrapers) publish the ip ranges they crawl from, anyone claiming to be googlebot (or whatever) but not in the ip range can safely be black holed.
• deleted 1mo ago
  [deleted]