2 ms·
Yeah, those selective hits scream custom scrapers or AI data hunters. To track 'em: - Parse logs: zcat access.log.* | awk '{print $1,$7}' | sort | uniq -c | so
by Jonhvmp 7mo ago
Yeah, those selective hits scream custom scrapers or AI data hunters. To track 'em:
- Parse logs: zcat access.log.* | awk '{print $1,$7}' | sort | uniq -c | sort -nr | head -20
Shows top IPs/paths. Whois suspicious ones.
- Add JS fingerprinting (canvas hashing, WebGL) to log real vs headless.
- Bait pages with unique content.
Set up alerts on anomalies. Caught some sneaky ones that way!