4 ms·
>Still to do: developing our own online ML based monitor which can detect abnormal user behaviour before it becomes a problem (ie. aggressive HTML scrapers, bur
by rovr138 6y ago
>Still to do: developing our own online ML based monitor which can detect abnormal user behaviour before it becomes a problem (ie. aggressive HTML scrapers, burp suite scans, etc.)
Search first. I would try something like,
detect irregular OR abnormal behavior internet OR requests OR databases
There are models for this and papers around it.
- lwansbrough 6y agoThe plan I have is to use an isolation forest with dimensions like requests per minute, unique pages per session, etc. and just present a recaptcha or inaccurate data when they're outside some threshold.
- rovr138 6y agoThat reminded me of this, https://blog.cloudflare.com/cleaning-up-bad-bots/ https://blog.cloudflare.com/cleaning-up-bad-bots/ Not sure if you had seen it. I like the approach. It'll still return a 200, but just slow them down. In a botnet, that's a pretty interesting approach instead of returning a captcha so they'll keep using the IP. Something to consider...