8 ms·

How hard is it to emulate an iPhone or Android so you can do this on a computer? I would think it would be a lot easier to do this with one PC instead of hundr

How hard is it to emulate an iPhone or Android so you can do this on a computer? I would think it would be a lot easier to do this with one PC instead of hundreds of tethered phones.

bongoman37 6y ago
I used to work on stopping this, its a cat and mouse game. Most datacenter IPs are already blacklisted, and this appears to be a low level fraud. But often it may be that you need physical actions also, e.g., an advertiser may pay for installing a game only if the player crosses the first level of a game. Either you build a bot which can do that or you use humans. Fraud goes all the way from simple operations where actual people in countries like India or Bangladesh are physically clicking on ads to sophisticated operations where bots generate human like traffic which is mixed with genuine traffic to avoid detection.
- bredren 6y ago
  How much did IP masking help in this situation versus the data analysis and identification of inauthentic use? Did you have any experience dealing with residential proxy networks? If so, can you give some examples of any mitigations to inbound requests from those connections?
  - bongoman37 6y ago
    IP masking was applied very early but it did cut down a large number of requests, I am not sure how many though. Next were filters that would block devices sending more than a few hundred requests a minute or so on. These would be usually bots or compromised devices. Then we would look at cuts of traffic where the characteristics were outside of acceptable parameters and block those. E.g., say a particular OS, OS version, device model, request country etc. This could be made much more granular. If you see too high a CTR or too low a conversion rate, or abnormal time between ad being displayed and being clicked on (fraud clicks had a tighter distribution between request and click as compared to genuine clicks) or any other weird stuff, you could block that cut of traffic. Also, we refrained from calling it fraud, the term was invalid traffic. This also accounted for things like someone double tapping on an ad instead of once, and so on. There is a good report by Alexander Tuzhilin [1] commissioned by Google when they were sued for charging for invalid traffic that might give you some more ideas. [1] https://googleblog.blogspot.com/pdf/Tuzhilin_Report.pdf https://googleblog.blogspot.com/pdf/Tuzhilin_Report.pdf
    - bredren 6y ago
      Thank you for this. I’m more interested in it from the perspective of anti-web scraping algorithms. I was curious, when you say block traffic, would you send 400s or silently just not record the crucial events? It seems like maybe tipping off automation would cause them to readjust. Perhaps they are not that determined?
      - bongoman37 6y ago
        Oh they were pretty determined. Sometimes when we would block traffic on particular cuts, we could see changes in traffic happening in as little as 30 mins which meant that fraudsters had automated systems responding to changes on our end. Things such as a single device ID sending hundreds of requests in a minute or blacklisted IPs would be just dropped silently, no point in even wasting bandwidth to show an ad. For cuts of traffic where the clicks etc were outside of predetermined ranges, it would again be blocked and dropped. In other cases, where fraud happened before we could block it, there was a flag called 'billable' which was set to False so the advertiser wasn't charged for it.
        bredren 6y ago
        I see. Who were the fraud operators affiliated with / what was their incentive? Isn’t it the publishers that would see value, or perhaps a competitor trying to disrupt marketing efforts? Do you see opportunity for innovation in traffic validation? Have you seen much in the way of training ML for this purpose?
- deleted 6y ago
  [deleted]
bserge 6y ago
It's doable, but I believe a bunch of old phones will come out cheaper (including on electricity cost) and more reliable (being proper devices in the eyes of ad networks, and having fewer crashes). A 4 core i7 CPU + 32GB of RAM will run 4-8 Android machines, maybe a couple more. You can get 10 Android phones for ~$700.
- saddestcatever 6y ago
  True. I think the economics of how cheap smartphones have gotten is shocking. Especially since you can acquire "barely working" phones. Broken camera? Not a problem. Broken audio jack? No worries. Cracked screen? Meh. No battery life? Doesn't matter one bit. I imagine it's pretty easy to acquire hundreds of barely working phone for cheap if you don't care about using them as a traditional mobile device.
  - Scoundreller 6y ago
    Or buy working devices and harvest the unneeded parts for resale. Can buy a used MacBook Air, sell the lcd assembly and keyboard/trackpad and already recover a big chunk of the cost. (If you needed an Apple device).
  - banana_giraffe 6y ago
    And once you start looking to acquire hundreds of phones, you can buy a lot of old phones on ebay or the like for something like $1 a phone or less. Most probably won't work, but at scale, you'll end up with enough that are good enough for something like a click farm.
- ac29 6y ago
  Why would an emulated phone need 4-8GB of RAM? A $70 phone probably only has 1 or 2 GB.
  - bserge 6y ago
    Worst case scenario overhead. Aka my normal situation, because I'm too lazy to use solutions better than Bluestacks on Windows :D
avidiax 6y ago
Rather difficult without an insider at Apple and Google. The fraud detection systems used are based on dozens of checks. I doubt even these real phones pass, since they likely have low uptime, they have never been in a landscape orientation or face down, few system settings are customized, etc. At least Android provides a public API to measure trustworthiness, but if you are in the business of selling clicks you may also be in the business of turning a blind eye to fraudulent activity that makes you money.
- bredren 6y ago
  I know a person who runs a bot on a social media app that does the follow / unfollow routine to boost an account. They've had this running for a few years. At first I assumed it was conservative enough to fly under the radar. However, now that this network is so big, the pattern of this fraudulent behavior must be unmistakable. It is so simple. The only conclusion I can draw is that the network is well aware of these bots, and so long as they don't draw negative attention, they are actually boosting "engagement" as the likes / follows do cause events that trigger people into revisiting the product--and ultimately reaching goals.
cyberlab 6y ago
> How hard is it to emulate an iPhone or Android Check out Correlium https://corellium.com/ https://corellium.com/ They operate in a sort of legal grey-area since you're not 'supposed' to emulate iOS so if you decide to use it, keep in mind it could all be shut down at a moments notice.
- monocasa 6y ago
  Better than legal grey area, as they won Apple's suit against them. https://www.washingtonpost.com/technology/2020/12/29/apple-corellium-lawsuit/ https://www.washingtonpost.com/technology/2020/12/29/apple-c...

How hard is it to emulate an iPhone or Android so you can do this on a computer? I would think it would be a lot easier to do this with one PC instead of hundreds of tethered phones.

bongoman37 6y ago

I used to work on stopping this, its a cat and mouse game. Most datacenter IPs are already blacklisted, and this appears to be a low level fraud. But often it may be that you need physical actions also, e.g., an advertiser may pay for installing a game only if the player crosses the first level of a game. Either you build a bot which can do that or you use humans. Fraud goes all the way from simple operations where actual people in countries like India or Bangladesh are physically clicking on ads to sophisticated operations where bots generate human like traffic which is mixed with genuine traffic to avoid detection.

bredren 6y ago
How much did IP masking help in this situation versus the data analysis and identification of inauthentic use? Did you have any experience dealing with residential proxy networks? If so, can you give some examples of any mitigations to inbound requests from those connections?
- bongoman37 6y ago
  IP masking was applied very early but it did cut down a large number of requests, I am not sure how many though. Next were filters that would block devices sending more than a few hundred requests a minute or so on. These would be usually bots or compromised devices. Then we would look at cuts of traffic where the characteristics were outside of acceptable parameters and block those. E.g., say a particular OS, OS version, device model, request country etc. This could be made much more granular. If you see too high a CTR or too low a conversion rate, or abnormal time between ad being displayed and being clicked on (fraud clicks had a tighter distribution between request and click as compared to genuine clicks) or any other weird stuff, you could block that cut of traffic. Also, we refrained from calling it fraud, the term was invalid traffic. This also accounted for things like someone double tapping on an ad instead of once, and so on. There is a good report by Alexander Tuzhilin [1] commissioned by Google when they were sued for charging for invalid traffic that might give you some more ideas. [1] https://googleblog.blogspot.com/pdf/Tuzhilin_Report.pdf https://googleblog.blogspot.com/pdf/Tuzhilin_Report.pdf
  - bredren 6y ago
    Thank you for this. I’m more interested in it from the perspective of anti-web scraping algorithms. I was curious, when you say block traffic, would you send 400s or silently just not record the crucial events? It seems like maybe tipping off automation would cause them to readjust. Perhaps they are not that determined?
    - bongoman37 6y ago
      Oh they were pretty determined. Sometimes when we would block traffic on particular cuts, we could see changes in traffic happening in as little as 30 mins which meant that fraudsters had automated systems responding to changes on our end. Things such as a single device ID sending hundreds of requests in a minute or blacklisted IPs would be just dropped silently, no point in even wasting bandwidth to show an ad. For cuts of traffic where the clicks etc were outside of predetermined ranges, it would again be blocked and dropped. In other cases, where fraud happened before we could block it, there was a flag called 'billable' which was set to False so the advertiser wasn't charged for it.
      - bredren 6y ago
        I see. Who were the fraud operators affiliated with / what was their incentive? Isn’t it the publishers that would see value, or perhaps a competitor trying to disrupt marketing efforts? Do you see opportunity for innovation in traffic validation? Have you seen much in the way of training ML for this purpose?
deleted 6y ago
[deleted]

bserge 6y ago

It's doable, but I believe a bunch of old phones will come out cheaper (including on electricity cost) and more reliable (being proper devices in the eyes of ad networks, and having fewer crashes). A 4 core i7 CPU + 32GB of RAM will run 4-8 Android machines, maybe a couple more. You can get 10 Android phones for ~$700.

saddestcatever 6y ago
True. I think the economics of how cheap smartphones have gotten is shocking. Especially since you can acquire "barely working" phones. Broken camera? Not a problem. Broken audio jack? No worries. Cracked screen? Meh. No battery life? Doesn't matter one bit. I imagine it's pretty easy to acquire hundreds of barely working phone for cheap if you don't care about using them as a traditional mobile device.
- Scoundreller 6y ago
  Or buy working devices and harvest the unneeded parts for resale. Can buy a used MacBook Air, sell the lcd assembly and keyboard/trackpad and already recover a big chunk of the cost. (If you needed an Apple device).
- banana_giraffe 6y ago
  And once you start looking to acquire hundreds of phones, you can buy a lot of old phones on ebay or the like for something like $1 a phone or less. Most probably won't work, but at scale, you'll end up with enough that are good enough for something like a click farm.
ac29 6y ago
Why would an emulated phone need 4-8GB of RAM? A $70 phone probably only has 1 or 2 GB.
- bserge 6y ago
  Worst case scenario overhead. Aka my normal situation, because I'm too lazy to use solutions better than Bluestacks on Windows :D

avidiax 6y ago

Rather difficult without an insider at Apple and Google. The fraud detection systems used are based on dozens of checks. I doubt even these real phones pass, since they likely have low uptime, they have never been in a landscape orientation or face down, few system settings are customized, etc. At least Android provides a public API to measure trustworthiness, but if you are in the business of selling clicks you may also be in the business of turning a blind eye to fraudulent activity that makes you money.

bredren 6y ago
I know a person who runs a bot on a social media app that does the follow / unfollow routine to boost an account. They've had this running for a few years. At first I assumed it was conservative enough to fly under the radar. However, now that this network is so big, the pattern of this fraudulent behavior must be unmistakable. It is so simple. The only conclusion I can draw is that the network is well aware of these bots, and so long as they don't draw negative attention, they are actually boosting "engagement" as the likes / follows do cause events that trigger people into revisiting the product--and ultimately reaching goals.

cyberlab 6y ago

> How hard is it to emulate an iPhone or Android Check out Correlium https://corellium.com/ https://corellium.com/ They operate in a sort of legal grey-area since you're not 'supposed' to emulate iOS so if you decide to use it, keep in mind it could all be shut down at a moments notice.

monocasa 6y ago
Better than legal grey area, as they won Apple's suit against them. https://www.washingtonpost.com/technology/2020/12/29/apple-corellium-lawsuit/ https://www.washingtonpost.com/technology/2020/12/29/apple-c...