3 ms·
I used to work on stopping this, its a cat and mouse game. Most datacenter IPs are already blacklisted, and this appears to be a low level fraud. But often it m
by bongoman37 6y ago
I used to work on stopping this, its a cat and mouse game. Most datacenter IPs are already blacklisted, and this appears to be a low level fraud. But often it may be that you need physical actions also, e.g., an advertiser may pay for installing a game only if the player crosses the first level of a game. Either you build a bot which can do that or you use humans.
Fraud goes all the way from simple operations where actual people in countries like India or Bangladesh are physically clicking on ads to sophisticated operations where bots generate human like traffic which is mixed with genuine traffic to avoid detection.
- bredren 6y agoHow much did IP masking help in this situation versus the data analysis and identification of inauthentic use? Did you have any experience dealing with residential proxy networks? If so, can you give some examples of any mitigations to inbound requests from those connections?
- bongoman37 6y agoIP masking was applied very early but it did cut down a large number of requests, I am not sure how many though. Next were filters that would block devices sending more than a few hundred requests a minute or so on. These would be usually bots or compromised devices. Then we would look at cuts of traffic where the characteristics were outside of acceptable parameters and block those. E.g., say a particular OS, OS version, device model, request country etc. This could be made much more granular. If you see too high a CTR or too low a conversion rate, or abnormal time between ad being displayed and being clicked on (fraud clicks had a tighter distribution between request and click as compared to genuine clicks) or any other weird stuff, you could block that cut of traffic. Also, we refrained from calling it fraud, the term was invalid traffic. This also accounted for things like someone double tapping on an ad instead of once, and so on. There is a good report by Alexander Tuzhilin [1] commissioned by Google when they were sued for charging for invalid traffic that might give you some more ideas. [1] https://googleblog.blogspot.com/pdf/Tuzhilin_Report.pdf https://googleblog.blogspot.com/pdf/Tuzhilin_Report.pdf
- bredren 6y agoThank you for this. I’m more interested in it from the perspective of anti-web scraping algorithms. I was curious, when you say block traffic, would you send 400s or silently just not record the crucial events? It seems like maybe tipping off automation would cause them to readjust. Perhaps they are not that determined?
- bongoman37 6y agoOh they were pretty determined. Sometimes when we would block traffic on particular cuts, we could see changes in traffic happening in as little as 30 mins which meant that fraudsters had automated systems responding to changes on our end. Things such as a single device ID sending hundreds of requests in a minute or blacklisted IPs would be just dropped silently, no point in even wasting bandwidth to show an ad. For cuts of traffic where the clicks etc were outside of predetermined ranges, it would again be blocked and dropped. In other cases, where fraud happened before we could block it, there was a flag called 'billable' which was set to False so the advertiser wasn't charged for it.
- bredren 6y agoI see. Who were the fraud operators affiliated with / what was their incentive? Isn’t it the publishers that would see value, or perhaps a competitor trying to disrupt marketing efforts? Do you see opportunity for innovation in traffic validation? Have you seen much in the way of training ML for this purpose?
- deleted 6y ago[deleted]