8 ms·

I understand that it is challenging for a third party like you to identify the bots. But Facebook has more than enough signal on each user clicking on an ad (fr

I understand that it is challenging for a third party like you to identify the bots. But Facebook has more than enough signal on each user clicking on an ad (friends, posts, location, 'non ad-click activity' to 'ad-click activity' ratio) to determine if that user is a bot. Looks like they are choosing to not do anything with this information. "FB is turning a blind eye to keep their revenue" seems to stand up to Occam's Razor.

rapind 14y ago
Couldn't agree more. It should be almost trivial for them to track down bots and flag bad accounts. I definitely wouldn't classify it as one of their harder problems.
- pmarsh 14y ago
  Not saying that's not the case, but you have no data or knowledge to back any of that up. I wouldn't assume a problem is easy or hard until I've got sufficient info on it. Too often I've heard "how could XYZ not have done this, fixed that?" Then have that same party sign on board to fix this "easy" problem and get themselves in a world of hurt.
  - ebtalley 14y ago
    data would be * IP addresses, one can assume a bot would only have a set of addresses they could use, barring botnets. * request patterns, ie: did the bot request css/js, etc * request timeframes * UA strings Sure, its a big data problem, but I can imagine that Facebook has solved these types of scenarios many times over.
    - zburt 14y ago
      What if you start a new Amazon EC2 spot instance (netting you a new IP address), start up Chromium in headless mode (say, using Xvfb), navigate to the website of choice, use mouse automation to start clicking around, click the ad, spend 5 minutes clicking around in a semi-choreographed pattern on the advertisee's website, and then shut down the instance -- only to repeat? Moreover, Amazon is always buying new IP subnets.
      - kirubakaran 14y ago
        The user from that new IP won't have any real human like history - photos shared and commented on over time etc.
      - geori 14y ago
        I dont count clicks from any amazonaws or ec2 hostnames on my site.
      - jonknee 14y ago
        It sounds like you don't need to go to that much hassle currently, but even that rigmarole is simple enough to combat. The user account should be real, the usage real (comments, photos, messages back and forth) and the friends also real. False positive spam Ids are OK, that will lower your revenue but won't constitute fraud with your customers. Put up a test for uses you think are spamming, the test they already do of identifying photos of your friends would be a good one. Large numbers of real looking fake accounts should be hard to keep up.
      - freeall 14y ago
        But why would you go through all that to click on ads? If I click on an ad for "Some Record Company" how does that make me money?
        aidos 14y ago
        It doesn't always need to make you money, sometimes you might just want it to cost your competitor money.
      - radicalbyte 14y ago
        Then pay Amazon for a list of their EC2 IPs, or obtain that information from a public source (i.e. RIPE, ARIN).
  - rapind 14y ago
    Well I do have at least some data to back that up. I know that they are a walled garden that keeps a ton of user data for every account (even the subset represented in Open Graph is significant). And it's with this knowledge that I make my conclusion. * Accounts subsisting of an unusual proportion of ad clicking activity compared to their other activities are likely bots. * New accounts with no history but high ad clicks are likely bots. Before a click is counted, check the source account for it's human rank (activity-to-clicks). If it's below a certain threshold (tuned over time) than don't bill your customer for it. The algo's to determine human rank can start fairly simple and become more complicated and accurate over time. In terms of their current sophistication I would definitely call this an easy problem from a tech standpoint. Scaling to millions of concurrent users? Hard. Attracting and keeping millions of users? Hard.
    - monkeypizza 14y ago
      Compare it to google's problem filtering out automatically generated splogs, though - it took them years to make progress against that. Real human social activity is a lot more random than real informational content in blogs - so fb's problem seems harder. Also, facebook banning accounts is a much stronger action than google just lowering the pagerank of suspected splogs. So overall I think fb's problem is harder than a currently google-unsolved problem.
      - jonknee 14y ago
        Facebook has total control of the users very action since joining Facebook. Much easier than coming across a random HTMl page and deciding on if it's real (whatever real means, YouTube comments are real but are very low signal but a database that runs itself could provide lots of signal).
      - Foy 14y ago
        He never said to ban the suspicious accounts, just don't charge companies for their clicks. The detection part aside, it's a simple fix. I just wonder why FB hasn't already. (Is it because it'd hurt their revenue when they need it the most?)
      - 14y ago
- pegmanm 14y ago
  This is a mostly solved problem though. Or at least there are people and products out there that do this specifically for you, not with a focus on bots alone as that is a bit of a edge case but for mobile devices. DeviceAtlas and WURFL are both products that I have used that attempt to have complete coverage of all UA strings in use in the wild even, the ones that attempt to spoof legitimate UA strings. While their focus is on device detection and the properties of these devices for mobile content adaptation, knowing which connections are from bots is just as important if you are going to alter what content you serve up based on UA. These product actively seek out new UAs and add them to their rule sets in a way that I can only assume will be more accurate over time than a home grown solution. FWIW I know that FB use at least one of these products for their device detection, so if you want to match what FB thinks (knows!?) this is where you would start.
johnbenwoo 14y ago
FB turning a blind eye towards shady a advertising practice because it generates revenue? See: Scamville http://techcrunch.com/2009/10/31/scamville-the-social-gaming-ecosystem-of-hell/ http://techcrunch.com/2009/10/31/scamville-the-social-gaming...
cookiecaper 14y ago
It's just as easy to invest in several fake profiles that act just like real users. Pay people to work from home maintaining fake accounts for you, posting stuff that seems real, interacting with your other fake accounts and celebrity/business pages, etc. Then, run bots with these accounts, click around on a subset of good links randomly, and throw in a reasonable ratio of ad clicks per account, then move on to the next shill account. This definitely takes non-trivial effort, but it's also definitely doable if you have a serious interest in perpetrating click fraud. While Facebook hypothetically could fight this form of click fraud, it'd likely be difficult and the method would be fragile/easily circumvented. Like most things, PPC advertising can only function in a world primarily populated with positive actors.
- mkup 14y ago
  It's much easier to guess a weak password than create a fake profile. As for everything else, I totally agree with you.
- simula67 14y ago
  I am unsure if Facebook provides a service like AdSense to motivate this kind of behavior. It would be interesting if to see how they could fight this type of behavior though. 1. Make the split uneven. For example for every dollar advertiser pays, only 30 cents goes to the page owner who received the "Likes" 2. Lobby to make these kinds of activity illegal. They are after all spamming activities 3. These kinds of fake account farms will usually have very few connections to outside world. That is, accounts that do not obey the "Six Degrees of seperation" should be flagged for human review. 4. Once reviewed enlist law enforcement help to track down the owners of the all the accounts in these farms. Painstaking work, but I guess the point is that advantage can lie with Facebook as opposed to the spammer in this game. I am not sure about the ethical implications though. Maybe this is why Google wants to succeed badly in the social space : extra spam fighting abilities.
- roel_v 14y ago
  You're basically saying 'there is a point where the line between actual user and paid click-frauder becomes blurry'. Which is true, but that point is way beyond what is profitable, which is enough - you 'only' need to make your system hard enough to beat to make it not worth the effort.
lifeisstillgood 14y ago
Just as a thought - facebook must be in a unique position to provide almost unbeatable captchas - just put up images of a known friend and four / forty random pictures - the one who chooses correct is likely the person logged in ignore this is fb already does it
- ars 14y ago
  They do in fact do that (or something like it) when you need to recover an account/password.
  - omellet 14y ago
    They also did this the first time I logged in from a foreign country.
lifeisstillgood 14y ago
Just as a thought - facebook must be in a unique position to provide almost unbeatable captchas - just put up images of a known friend and four / forty random pictures - the one who chooses correct is likely the person logged in ignore this is fb already does it
lance0 14y ago
I suspect even that signal is not trustworthy. Given that its not 100%, what fraction of the 900+ million Facebook users are actually users? Twitter seemed to be totally plagued with that at one point anyway. What's the grey area look like? What's the chance of falsely identifying a real user as a fake one? Rhetorical question. Not sure FB could even answer them accurately.

rapind 14y ago

Couldn't agree more. It should be almost trivial for them to track down bots and flag bad accounts. I definitely wouldn't classify it as one of their harder problems.

pmarsh 14y ago
Not saying that's not the case, but you have no data or knowledge to back any of that up. I wouldn't assume a problem is easy or hard until I've got sufficient info on it. Too often I've heard "how could XYZ not have done this, fixed that?" Then have that same party sign on board to fix this "easy" problem and get themselves in a world of hurt.
- ebtalley 14y ago
  data would be * IP addresses, one can assume a bot would only have a set of addresses they could use, barring botnets. * request patterns, ie: did the bot request css/js, etc * request timeframes * UA strings Sure, its a big data problem, but I can imagine that Facebook has solved these types of scenarios many times over.
  - zburt 14y ago
    What if you start a new Amazon EC2 spot instance (netting you a new IP address), start up Chromium in headless mode (say, using Xvfb), navigate to the website of choice, use mouse automation to start clicking around, click the ad, spend 5 minutes clicking around in a semi-choreographed pattern on the advertisee's website, and then shut down the instance -- only to repeat? Moreover, Amazon is always buying new IP subnets.
    - kirubakaran 14y ago
      The user from that new IP won't have any real human like history - photos shared and commented on over time etc.
    - geori 14y ago
      I dont count clicks from any amazonaws or ec2 hostnames on my site.
    - jonknee 14y ago
      It sounds like you don't need to go to that much hassle currently, but even that rigmarole is simple enough to combat. The user account should be real, the usage real (comments, photos, messages back and forth) and the friends also real. False positive spam Ids are OK, that will lower your revenue but won't constitute fraud with your customers. Put up a test for uses you think are spamming, the test they already do of identifying photos of your friends would be a good one. Large numbers of real looking fake accounts should be hard to keep up.
    - freeall 14y ago
      But why would you go through all that to click on ads? If I click on an ad for "Some Record Company" how does that make me money?
      - aidos 14y ago
        It doesn't always need to make you money, sometimes you might just want it to cost your competitor money.
    - radicalbyte 14y ago
      Then pay Amazon for a list of their EC2 IPs, or obtain that information from a public source (i.e. RIPE, ARIN).
- rapind 14y ago
  Well I do have at least some data to back that up. I know that they are a walled garden that keeps a ton of user data for every account (even the subset represented in Open Graph is significant). And it's with this knowledge that I make my conclusion. * Accounts subsisting of an unusual proportion of ad clicking activity compared to their other activities are likely bots. * New accounts with no history but high ad clicks are likely bots. Before a click is counted, check the source account for it's human rank (activity-to-clicks). If it's below a certain threshold (tuned over time) than don't bill your customer for it. The algo's to determine human rank can start fairly simple and become more complicated and accurate over time. In terms of their current sophistication I would definitely call this an easy problem from a tech standpoint. Scaling to millions of concurrent users? Hard. Attracting and keeping millions of users? Hard.
  - monkeypizza 14y ago
    Compare it to google's problem filtering out automatically generated splogs, though - it took them years to make progress against that. Real human social activity is a lot more random than real informational content in blogs - so fb's problem seems harder. Also, facebook banning accounts is a much stronger action than google just lowering the pagerank of suspected splogs. So overall I think fb's problem is harder than a currently google-unsolved problem.
    - jonknee 14y ago
      Facebook has total control of the users very action since joining Facebook. Much easier than coming across a random HTMl page and deciding on if it's real (whatever real means, YouTube comments are real but are very low signal but a database that runs itself could provide lots of signal).
    - Foy 14y ago
      He never said to ban the suspicious accounts, just don't charge companies for their clicks. The detection part aside, it's a simple fix. I just wonder why FB hasn't already. (Is it because it'd hurt their revenue when they need it the most?)
    - 14y ago
pegmanm 14y ago
This is a mostly solved problem though. Or at least there are people and products out there that do this specifically for you, not with a focus on bots alone as that is a bit of a edge case but for mobile devices. DeviceAtlas and WURFL are both products that I have used that attempt to have complete coverage of all UA strings in use in the wild even, the ones that attempt to spoof legitimate UA strings. While their focus is on device detection and the properties of these devices for mobile content adaptation, knowing which connections are from bots is just as important if you are going to alter what content you serve up based on UA. These product actively seek out new UAs and add them to their rule sets in a way that I can only assume will be more accurate over time than a home grown solution. FWIW I know that FB use at least one of these products for their device detection, so if you want to match what FB thinks (knows!?) this is where you would start.

johnbenwoo 14y ago

FB turning a blind eye towards shady a advertising practice because it generates revenue? See: Scamville http://techcrunch.com/2009/10/31/scamville-the-social-gaming-ecosystem-of-hell/ http://techcrunch.com/2009/10/31/scamville-the-social-gaming...

cookiecaper 14y ago

It's just as easy to invest in several fake profiles that act just like real users. Pay people to work from home maintaining fake accounts for you, posting stuff that seems real, interacting with your other fake accounts and celebrity/business pages, etc. Then, run bots with these accounts, click around on a subset of good links randomly, and throw in a reasonable ratio of ad clicks per account, then move on to the next shill account. This definitely takes non-trivial effort, but it's also definitely doable if you have a serious interest in perpetrating click fraud. While Facebook hypothetically could fight this form of click fraud, it'd likely be difficult and the method would be fragile/easily circumvented. Like most things, PPC advertising can only function in a world primarily populated with positive actors.

mkup 14y ago
It's much easier to guess a weak password than create a fake profile. As for everything else, I totally agree with you.
simula67 14y ago
I am unsure if Facebook provides a service like AdSense to motivate this kind of behavior. It would be interesting if to see how they could fight this type of behavior though. 1. Make the split uneven. For example for every dollar advertiser pays, only 30 cents goes to the page owner who received the "Likes" 2. Lobby to make these kinds of activity illegal. They are after all spamming activities 3. These kinds of fake account farms will usually have very few connections to outside world. That is, accounts that do not obey the "Six Degrees of seperation" should be flagged for human review. 4. Once reviewed enlist law enforcement help to track down the owners of the all the accounts in these farms. Painstaking work, but I guess the point is that advantage can lie with Facebook as opposed to the spammer in this game. I am not sure about the ethical implications though. Maybe this is why Google wants to succeed badly in the social space : extra spam fighting abilities.
roel_v 14y ago
You're basically saying 'there is a point where the line between actual user and paid click-frauder becomes blurry'. Which is true, but that point is way beyond what is profitable, which is enough - you 'only' need to make your system hard enough to beat to make it not worth the effort.

lifeisstillgood 14y ago

Just as a thought - facebook must be in a unique position to provide almost unbeatable captchas - just put up images of a known friend and four / forty random pictures - the one who chooses correct is likely the person logged in ignore this is fb already does it

ars 14y ago
They do in fact do that (or something like it) when you need to recover an account/password.
- omellet 14y ago
  They also did this the first time I logged in from a foreign country.

lifeisstillgood 14y ago

lance0 14y ago

I suspect even that signal is not trustworthy. Given that its not 100%, what fraction of the 900+ million Facebook users are actually users? Twitter seemed to be totally plagued with that at one point anyway. What's the grey area look like? What's the chance of falsely identifying a real user as a fake one? Rhetorical question. Not sure FB could even answer them accurately.