8 ms·
Inside Google's Secret War Against Ad Fraud
- ChuckMcM 11y agoThey make it sound a lot sexier than it is. When Blekko was serving up search pages we had over 200,000 machines at any given time identified as botnet/fraud sources. Perhaps it was because we were providing search results to "tier n" search portals that we got to see more of the folks trying to scam the advertisers but we got really really good at not serving up ads or counting clicks on robots. Something ad publishers really appreciated. If you're serving up SERPs there are lots of interesting things you can do, like make white space clickable but not go anywhere, (humans click on the ads, bots click on the giant click box.) Serve up an invisible "first ad" on the top left, center, and right of the page. Tons of these robots just click the top ad they think they find. When IP's in europe, brazil, mexico and the Ukraine all search for the exact same english string and always click on the third ad down, well you can just put all of that addresses in the equivalent of an Ad "virtual kingdom" (they think they click but they don't go anywhere useful). Lots of funny stories, my favorite was sending Google traffic to Google ads and having Google tell us it was fraudulent traffic. That was pretty funny.
- cromwellian 11y agoMy guess is, given the scale, and amount of money on the line, the attacks against Google are far more sophisticated than just bots blindly clicking on ads.
- ChuckMcM 11y agoI would hope so! But even a place like Ask.com is clearly a couple million dollars a day in ad revenue.
- sanswork 11y agoI wrote an ad system that was in part used to backfill Ask in a few countries and even with just backfill I was amazed at how much traffic they were still doing. Though I'm pretty sure most of it was toolbar traffic.
- deleted 11y ago[deleted]
- spyder 11y agoYes, there is so much money in it that it's very likely there are bots with an AI trained on real human behavior samples, making it hard to distinguish from real clicks.
- ChuckMcM 11y agoActually easier than this, a bit of spyware that looks at your searching/clicking habits over a very long period and then sends that to a big mixer which then replays the same search/click with the same timing and mouse motion. I remember seeing a World of Warcraft automation script being demonstrated and thinking that it was probably used to abuse ad networks.
- SagelyGuru 11y agoThere was a time when Google was a likeable company creating good search engines. Now it is all about battles over the ads. In what sense is that still beneficial to an ordinary web surfer? Is this an inevitable progression or is it a matter of wrong priorities?
- danmaz74 11y agoIf you want good search engines without having to pay to use them, then those search engines need to run ads...
- jacquesm 11y agoIs there a good ad-free search engine that I can pay for?
- jfuhrman 11y agoThe issue with search engines is that they tend to get very good by using clickthrough data, so the more you use something the better it gets. Indexing and ranking algorithms can only do so much. The interesting question is that does Google own the click through data or does the user? Does Google exclusively own the fact that you clicked on the 4th link in Google leading to php.net when you searched for 'php'? Or can the user or government get this data to alternate search engines for better competition just like they forced Microsoft to open up Office file formats and SMB protocol docs to their rivals? If you remember the big deal about the optional Bing bar in IE uploading the search term and clicked URL in Google search results to Bing it looks like many people on here think Google exclusively owns the clickstream data, and it's not the users' to do what it wants. This is the reason that a paid search engine won't do very well, just like a paid social network wouldn't, because both rely on a lot of people using it to seed it and a paywall will hurt.
- ChuckMcM 11y agoIt is the interesting question of our times isn't it? I did the calculation for Blekko at one time and a couple of million paid subscribers could have floated the base search engine. It is hard to find people to subscribe to things on the internet though unless they are MMO games :-)
- frik 11y agoSad of the demise of Blekko search engine. It's now part of IBM to build their inhouse "freebase" style data source for Watson AI, as Google closed freebase.com I wonder what DuckDuckGo will do, as their primary search source the Yahoo BOSS API got more expensive in June 1, 2015: https://developer.yahoo.com/boss/search/#pricing https://developer.yahoo.com/boss/search/#pricing ($1.80 per 1000 Queries; the free tier is no more)
- Scoundreller 11y ago$1.80/CPM should be absorbable with ad revenue, but I fear the average DuckDuckGoer runs ad blockers, blocks javascript, runs Ghostery and doesn't click on ads very often... Looks like the old pricing was $0.80/CPM. Any idea if they have a direct contract with DDG?
- cpeterso 11y ago> When IP's in europe, brazil, mexico and the Ukraine all search for the exact same english string and always click on the third ad down, well you can just put all of that addresses in the equivalent of an Ad "virtual kingdom" (they think they click but they don't go anywhere useful). How much of a server drain is the "virtual kingdom"? Could you point those trap ads to some host or IP address that takes a long time for the client to reach, offloading the wasted time from your server to their client?
- deleted 11y ago[deleted]
- downandout 11y agoI'm not sure why the criminals are bothering. AdSense clicks earn almost nothing in my experience. I guess they could be making money by charging businesses to click on their competitors' ads and drain their budgets, but there are only so many companies that are willing to pay to engage in clearly illegal activity like that. The primary beneficiary of the fraudulent clicks that are good enough to evade Google's detection is....Google.
- sanswork 11y agoAdsense might not pay a lot for most things but there are still a lot of high paying keywords you can target and certainly enough money to be made to draw out bad actors.
- Scoundreller 11y agoYep. I've built admittedly high-margin and low-volume sites, but I've averaged about $0.75/click on Adsense for several years. I haven't even done any A/B testing either. Now if I could only ratchet up the volume...
- ISL 11y agoNot in the long run. If Google ad customers aren't seeing value for their ad spend, they'll take their dollars elsewhere.
- Scoundreller 11y agoI'm not so sure. I always figured the cost per click would drop accordingly. If fraud is 50%, advertisers would only bid 50% for a click/impression, and it would be a wash in the end for the advertiser and Google. I think the real hit is against content creators who aren't able to extract the full value of their work, instead they split the revenue pool with a bunch of adspam sites.
- antsar 11y agoCouldn't Google keep cost-per-click artificially high, benefiting themselves at the expense of advertisers?
- deleted 11y ago[deleted]
- sprw121 11y agoI work in adtech, specifically on the bidding side on real-time click forecasting and ad-pricing. I have to say that compared to other ad exchanges. AdX (google's ad exchange) is far and ways ahead of any other ad exchange in minimizing the the amount of fraudulent traffic that we as ad buyers get exposed to. I did them serious props.
- enjoylife 11y agoBesides the obvious scale differences, what have you seen to be big differentiators between google and other ad exchanges?
- asdf99 11y agoalso being an volume ad middle man in the industry, I can bet what the top comment was referring was price. adx is the cheapest, because it's the worse. your ad will be seen by users in spots previously occupied by "the banks hate this guy" kind of click bait ads. which have ridiculous ctr! everyone I run experiments there I keep getting calls from clients to take their ads out of some and some sites before brand sees it. it's a nightmare.
- deleted 11y ago[deleted]
- sprw121 11y agoNo, not price, although they are pretty cheap and run a fair auction system (unlikely other exchanges...looking at you facebook/appnexus). The CTR on adx is actually one of the better ones, and they tend to have less junk inventory.
- sprw121 11y agoIn terms of fault, we're doing what we can to filter it out when it hits us, but nothing particularly sophisticated yet. On other exchanges, our filters filter out some significant percentage of traffic, but the same filters applied on Adx don't get triggered often. The click and conversion behavior of ads shown number seem to suggest that this is due to google actually doing a really good job of filtering the fraud before it hits us.
- kefs 11y agoWinLister, from the article.. http://www.nirsoft.net/utils/winlister.html http://www.nirsoft.net/utils/winlister.html
- just_observing 11y ago"Sasha and a number of his fellow Google employees asked to be referred to by their first names, saying they were concerned for their safety” "The rapport between original team members and their new peers was apparent as the crew gathered at the Craft Beer Co., a homey London pub" and they show a picture of some of the team. Not too worried then.
- yvoschaap2 11y agoTo me this reads as one big Google propaganda ad (by inviting a journalist).
- CodingGuy 11y agoGoogle style - like they do it with their webspam team.
- bootload 11y ago"Thanks to Google's massive size, the blueprint can then be overlaid on top of Google's wealth of impression data to find chunks of traffic that match up." 'z00clicker vs. normal click density' Any idea how/where the *normal click density described is sourced from?
- shabble 11y agoMaybe they have a sample user group who are equipped with some keyboard/mouse logging type system that can report to them? They could even do it to all (or some subset) of visitors via Google Analytics[1] to report the (x,y) position when click occurs. It wouldn't surprise me if an awful lot of the internal use of GA is for ad-fraud detection. I recall way back in the day some ad networks were using imagemaps[2] instead of plain links because they'd report an (x,y) postition relative to the image, and could probably filter out the really dumb clickbots that would hit the precise pixel centre of the element every time. [1] https://developers.google.com/analytics/devguides/collection/gajs/eventTrackerGuide https://developers.google.com/analytics/devguides/collection... [2] https://en.wikipedia.org/wiki/Image_map https://en.wikipedia.org/wiki/Image_map
- bootload 11y ago"They could even do it to all (or some subset) of visitors via Google Analytics[1] to report the (x,y) position when click occurs. It wouldn't surprise me if an awful lot of the internal use of GA is for ad-fraud detection." This is what I'd suspected, randomised sampling.
- kevin_thibedeau 11y agoI wish they would get off their butts and wage war against the referral spam problem that has made analytics useless for low traffic sites. The current filter system is not good enough to combat them. They need to provide a special filter with a curated blacklist that automatically removes all past data logged from the spam sites.
- Scoundreller 11y agoI would say Google's move toward HTTPS by default made analytics useless. You can't tell what search terms were used to arrive at your site. For my low-traffic sites, I would mine them by hand to figure out what people really wanted, and then develop content around that. I don't understand why Google would not want to incentivize that.
- filereaper 11y agoIt looks like taking down botnets is important to all search engines, here's a old news article about Bing taking down a botnet[0]. Similar to Bing, does Google also pursue and do an actual takedown of these botnets? Most of the article was about showing off how nasty these exploits are, nothing about taking them down. [0] http://www.cbc.ca/news/technology/huge-zeroaccess-botnet-disrupted-by-microsoft-1.2453707 http://www.cbc.ca/news/technology/huge-zeroaccess-botnet-dis...
- shabble 11y agoI'm reminded of 'Google Will Eat Itself'[1], which purportedly used this technique to purchase stock in Google, and also pay for additional ad-clicks, with the goal of buying out Google entirely from received ad revenue. I don't know (or really believe) that it ever actually operated in the way described though. More of an art/thought project. [1] http://www.gwei.org/pages/diagram/diagram.html http://www.gwei.org/pages/diagram/diagram.html
- ovi256 11y agoI remember when that was published and commenters laughed at it. I specifically remember a sophisticated explanation about how GWEI would never work because the second derivative of their GOOG would never something or another. Today GWEI owns $400k in GOOG.
- ksk 11y agoWhile fraud of any kind is certainly bad, I don't particularly feel enthused about cheering for the advertising industry as the "good guys".
- asdf99 11y agoGoogle removed itself from the IAB's working group on safe frame, which fought for user and publisher safety. among things like preventing the ad from knowing what's on the page it's served on it also handle if the ad is in view, preventing most of those bot net traffic that uses 1x1 iframes or insert ads with low zindex on the page. the only vulnerabilities left with safeframes are giving the user a executable when they click on the ads (nothing can prevent that) and flash exploits (although the most common exploits are blocked fine, it's still vulnerable when it's a full control bug in flash) but Google got out when they realized it wouldn't give as much info from its adsense network of third party sites as they are used to with insecure adwords (when it shows display ads)
- TazeTSchnitzel 11y agoGoogle's ad network relies on knowing about the page the ad is embedded in both for targeting (AdSense) and apparently fraud prevention.
- asdf99 11y agoexactly. it is a trade off. user privacy vs price on expanding inventory. they choose the later
- z3t4 11y agoGuess how many out of 1000 visitors from my last AdWords Content Campaign loaded the favicon.ico ? And how many of them do you think signed up to to my service!? Answer: 0/1000 (nada, zero) Adwords for search worked much better though (real users). I also tried Google ad-sense for some of my websites, but the quality of ads are very bad and no-one clicks on them. Around ten years ago, Google ads used to be "the shit" (everyone used it) and you where able to both earn money and get quality traffic. Have the publishers turned to other market places or is my current experience what the world wide web ad market looks like today!?