9 ms·
Advertisers scramble as ‘non-human traffic’ eats up online budgets
- tn13 12y agoThis is a problem I have seen first hand. But this is not really as big a problem as it is made out to be. The real issue here is that it is hard to predict of the given impression is a bot impression or a genuine impression. As a consequence any CPM based dealing is difficult. Difficult only means falling CPMs. That is what I have observed. The advertisers will not have to put in more efforts to measure conversions and accordingly bid for the traffic.
- mianos 12y agoI am also working in ad yield analytics, so while I wait for a DFP report to run, I'll also say, it's also in the network's interest to address this issue. In the meantime we'll just see more CPA line items and a drop in price for CPD and CPM. Being an auction, I'm sure Appnexus already sees this. When the issue is addressed we'll see a rise in CPM that reflects the increase in value with less bots. There is money in fixing it so I'm not calling the end of the on-line advertising world like the article, which is written by a general reporter not someone in the field (Cost Per Mile? oh really?)
- shostack 12y agoJust wait until cross-channel attribution (particularly dynamic attribution vs. fixed model which is more commonly available) really takes off. CPA-based buys are going to come under a LOT more scrutiny when Joe Advertiser suddenly realizes that his retargeting campaign didn't actually add a ton of value to the path for the vast majority of his conversions, especially if he's giving <gasp> 100% credit to view-throughs.
- brandnewlow 12y agoIt's funny how much business we've won just by making it easy for clients to take control over their view through attribution settings and counting.
- shostack 12y agoThat's awesome that PerfectAudience does that, and more and more DSPs are definitely waking up to that as well which is great. What do you do to help inform what the settings should be? Unless an advertiser has a solution like VisualIQ, Convertro, Adometry, etc. (and they've all been bought up so who knows what will happen to them), or GA Premium, they likely don't have a dynamic attribution solution in place to help inform them as to what those values should be and how they are shifting over time (because in reality those weights are anything but static). I really wish DSPs did more in the way of thought leadership on how to properly value view-throughs and other ways of measuring contribution as view-throughs are the majority of what they drive, and that metric is coming under increasing scrutiny (as it should).
- jsonne 12y agoI'm in advertising too. What I'm really hoping comes out of this is greater transparency in inventory. Give me greater control over my blacklist and I'll be a lot happier.
- jleader 12y agoActually, the article (correctly) expanded the acronym as cost-per-mille (note the 2 L's). Mille is latin for thousand.
- jbob2000 12y ago"Wah, I can't advertise using old methods and might have to come up with something new. Waaah"
- nkozyra 12y agoWhile I understand the downvotes, the sentiment here is valid in the sense that there's been so little innovation in advertisement for the web - even mobile ads are anachronistic. The entirety of innovation has been sizes & styles and tracking and not on presentation and usability that's complementary with content. It's not a 'you made your bed' situation, but not wholly removed from it, either.
- vinbreau 12y agoI envision a future where a significant share of the advertising market is comprised of automated ad-bots pitching services and wares to automated web-crawlers. There's something obtusely philosophical about that and what it says about us, though I am not sure what that is.
- arbitrage 12y agoWilliam Gibson could probably give you some ideas on what that says about us. He started writing about this fictionally decades ago.
- wernercolangelo 12y agoOr Stanislaw Lem
- pavel_lishin 12y agoCharles Stross, too.
- pessimizer 12y agoOr Frederick Pohl and Cyril Kornbluth https://en.wikipedia.org/wiki/The_Space_Merchants https://en.wikipedia.org/wiki/The_Space_Merchants (Or Alvin Lucier: https://en.wikipedia.org/wiki/I_Am_Sitting_in_a_Room https://en.wikipedia.org/wiki/I_Am_Sitting_in_a_Room)
- cvburgess 12y agoI wouldn't be surprised to find that the larger firms are crawling their own ad platforms. When ads comprise 90% of your revenue, it's hard to imagine that someone hasn't gamed the system.
- deleted 12y ago[deleted]
- perlgeek 12y agoThere are two pretty obvious approaches: only pay per conversion, or accept that not all views are human. Just like when you advertise in a classical news paper, you only know how many copies are typically sold, not how many humans actually look at your ad.
- benologist 12y agoThis is what it's called: http://en.wikipedia.org/wiki/Cost_per_action http://en.wikipedia.org/wiki/Cost_per_action It's been about for a long time.
- nkozyra 12y agoExcept not all conversions are human either. That's the "new" problem.
- shostack 12y agoIf only CPA were that simple. Case in point... - You work with 3 CPA networks - Each has their own tracking tag that their terms require payment be based off of - A conversion comes in and each network registered an impression (or heck, even a click) before the visitor in question signed up. Who gets the credit? Well, if you don't have an agreement that states billing is done off of numbers from YOUR ad server, where you can dedupe and set an attribution model that makes sense for your particular business, the answer is that all of them will. So you are paying 3x when realistically odds are that not all of them should get full credit. Hell, that doesn't even factor in other channels like organic, social, etc. that all likely played a role as well.
- bthornbury 12y agoI've wondered about this before if advertising was just designed to be a big negative sum game for advertisers. Interestingly if these same bots were capable of making real purchases and were run by ad companies this behavior may be sustainable because some people are making money. Almost like wall street.
- meric 12y agoAssuming an average consumer will spend a constant number of dollars per year - advertising would indeed be a big negative sum for advertisers. Advertisers would make the same number of dollars, except having had to spend on advertising. The problem is those who do not advertise generally lose out to those who do.
- etchalon 12y ago"Half the money I spend on advertising is wasted; the trouble is I don't know which half." http://en.wikipedia.org/wiki/John_Wanamaker http://en.wikipedia.org/wiki/John_Wanamaker
- solistice 12y agoWhilst that quote is about targeting and not fraud, it fits rather nicely. It's actually attributed to a number of people.
- goshx 12y agoBesides the non-human traffic, most online ads are created to not be seen. I was even thinking about creating a blog post to call out the marketers out there to change this, but I am too lazy... specially to make them make more money. I am so used to how ads that cover the entire page work that I manage to pass by them without even knowing which brand was advertising there. Ads that force you to watch a video for X seconds, but in those X seconds they don't even show a logo or something, are money wasted indeed, again, IMHO. Another example, ads that follow me. Oh god how I hate these. I visit a website, usually for a product that I already own, and the freaking ad from that company follows me wherever I go. This is money wasted. Another example, I use Waze almost everyday to figure out the best route for my comute. My 4" screen phone sits there right on top of the radio. Whenever I stop the car, an ad pops up. But they make this ad so tiny that if I need to know what brand or product it is for, I have to pick my phone and look closer. This is money wasted. The online advertising system is pretty broken... can I fix it?
- throwaway20148 12y ago> Another example, ads that follow me. Oh god how I hate these. I visit a website, usually for a product that I already own, and the freaking ad from that company follows me wherever I go. This is money wasted. Wasted on you, probably not on the majority.
- duaneb 12y agoDoes the average person like to purchase an item they already own if they see an advertisement for it? Of course not.
- blatherard 12y agoThere's a lot of room for innovation in this space. Some friends (well, clients) of mine have a startup (Unlockable, https://www.unlockable.com/ https://www.unlockable.com/) that embeds advertising-based games into other games. The level of interactivity is way more than a bot could handle.
- goshx 12y agoSo are bots playing games as well? I mean, are they also having ad fraud inside games?
- arbuge 12y agoI've been in the online advertising world for 10 years and it still befuddles me to see that most ad dollars still haven't switched to cost-per-action (CPA), i.e. only paying for actual conversions. One big benefit of advertising online after all is that tracking conversions and their referrers is much easier than with offline.
- eli 12y agoCPA simply doesn't make sense for all ad campaigns. When Lexus buys ads on a financial news site it's for branding -- they want to get their logo in front of their target demographic. The ads aren't optimized for clicks or leads -- sometimes there isn't even a clear call to action.
- deleted 12y ago[deleted]
- hyperpape 12y agoYup, but then why bother with CPM? Just buy display ads for a fixed price in publications they believe their customers might read. Sure, it's not an improvement over what they did in the 50s but they don't have to worry about fraud.
- brandnewlow 12y agoThose ads are freakishly expensive. Like 10x more expensive than buying ads on FB/Google/Twitter.
- eli 12y agoMy very biased opinion is that you get what you pay for.
- brandnewlow 12y agoIn that case you're paying the commissions on the expensive sales people needed to take your order over the phone.
- dangrossman 12y agoBack in July, hundreds (thousands?) of sites noticed a surge in traffic in their analytics. Hundreds to thousands of additional visits per day above their normal traffic rates. Curiously, all this traffic identified as Internet Explorer 7 on various version of Windows, but each visit came from a different residential IP, and viewed only one page per session. Eventually the common thread between all these sites was identified -- they had retargeting tags from sites like AdRoll: http://www.thesempost.com/adroll-retargeting-bot-attack-behind-ie7-traffic-surges/ http://www.thesempost.com/adroll-retargeting-bot-attack-behi... For months now, huge botnets have been producing huge amounts of fake web traffic, picking up retargeting cookies in order to later be served ads and have whoever's behind the network paid for those views. That surge has never ended. It's only grown (including becoming less easy to spot), yet nobody seems to be talking about it anymore, including the retargeting companies. I'd love to know who's behind it and how they control so many computers.
- throwaway20148 12y agoSometimes this stuff isn't even bot traffic, it's traffic purchased from someone who fulfills by incentivizing people who play online games with virtual currency for viewing videos or clicking through to sites that serve the buyers ads. I presume they've also got into the retargeting capture game, too.
- JeremyMorgan 12y agoYup. Many facebook games have a "watch this video" or "visit these sites" in an iframe in exchange for extra points, more seeds, a better pool stick, etc. Just as useless as bot traffic.
- hagbardgroup 12y agoSV will not care about this until another 2000-level crash. Then, they will be forced to care, because the revenue will stop coming in. Until then the community will do the see no evil / hear no evil / speak no evil routine.
- ljk 12y ago
- justcommenting 12y agothere's a certain irony when advertisers--whose intention is to manipulate the preferences/behaviors/purchases of consumers--cry foul about their manipulation systems being manipulated
- ForHackernews 12y agoGood. Maybe they'll stop throwing so much wasted money at Google.
- roymurdock 12y agoAdvertising in its current form is broken. I was recently in a class where we performed a case study on Google. The professor asked if anyone had ever clicked on an ad. One kid out of 60 had clicked on ad for cheap protein supplements, but hadn't bought any. Everyone else scoffed, pointed out their use of AdBlock, and cited their gullible parents as the only people they knew who might ever possibly accidentally click on an ad. Furthermore, the strong majority of our class viewed targeted ads on Facebook and Gmail as extremely creepy and unconvincing. There are two ad models that work in my opinion: 1. Native advertising: Ads disguised as useful content. This works for changing people's perceptions, especially if its done through official or generally impartial channels, such as news outlets. People think they're getting honest information about BP's charitable acts and Coke's health benefits, but they're really being lied to. This doesn't generate any direct sales, but it can really influence consumer sentiment on different brands which can have a massive impact on sales indirectly. This method is extremely sneaky and unethical, and that's probably why it works. 2. Snapchat's new model: Opt-in, high quality ads. I liken these ads to movie trailers. They are well-crafted and curated, and you only have to watch them if you want to. This way, people who click on them are already intrigued and have committed to watching the ad, making for a more engaged and curious user base. This method is still being developed but I can see it become big in the next couple of years.
- Silhouette 12y agoEveryone else scoffed, pointed out their use of AdBlock If your entire class was running AdBlock, then your sample is far from representative of the general population. There are two ad models that work in my opinion The thing is, opinions don't matter in this game, facts do. Do you really think major brands like Coca Cola or McDonalds or Ford spend such astronomical sums of money on advertising without being pretty good at knowing what returns they get? You might think that no-one really buys a car because of the product placement where the successful business owner drives that model in their favourite TV show, but you'd be wrong.
- thaumasiotes 12y ago> You might think that no-one really buys a car because of the product placement where the successful business owner drives that model in their favourite TV show That would be an unusual viewpoint. People love identifying with their heroes -- I don't think anyone considers that a secret. Related: http://basicinstructions.net/basic-instructions/2014/10/26/how-to-explain-why-you-want-what-you-want.html http://basicinstructions.net/basic-instructions/2014/10/26/h...
- Animats 12y agoSo now 25% to 50% of impressions go to ad bots. It's known that most ad clicks from about 10% of human users, who will click on anything but buy little. The real winner in this is Amazon. Amazon doesn't pay anybody when someone clicks on a product listing. They have to buy. Same for eBay. So there's no point in 'bots clicking on those sites. Online ad rates have been dropping for some time now. Most of the ads run by Google Adsense (the ones on third party sites) are there because Google pushes advertisers hard to use that feature. Most of the conversions from Google Ads come from the ones on search results, which are displayed when the user is looking for something. Those are useful. Most other forms of advertising are interruptions for users interested in something else. The classic tactic ad-based companies try as ad prices drop is increasing the ad density. That's called "pulling a Myspace". It didn't work for Myspace, which tanked. Twitter seems to be headed that way - usage up, profits not too good. The way this is going, online advertising is going to have the conversion (to a sale) rate of spam - one in thousands.
- rebootthesystem 12y agoWe stopped all advertising on Facebook. It's a mess.
- jsonne 12y agoReally? I've had quite a bit of success with facebook for my clients. What didn't you like about it?
- rebootthesystem 12y agoFake non-responsive non-engaging likes to pages and more. See this, it's very true: http://www.youtube.com/watch?v=oVfHeWTKjag http://www.youtube.com/watch?v=oVfHeWTKjag
- jsonne 12y agoI have seen that. That mostly has to do with boost posts though. We only use custom tailored audiences for our ads and it works quite well. Albeit there's a lot of data scrubbing.
- rebootthesystem 12y agoIt really goes beyond that. Facebook wants to own your audience and that is fundamentally wrong. Let's say I spend $100K advertising on FB to get people a my FB page or group. In order to now be able to reach them I have to constantly pay FB through boosts and further ads. Why? Because they've installed a mechanism through which they limit the reach of every single post you make on your page. And, if a percentage of your likes are fake or non-responsive for various reasons (they liked the picture on your ad, couldn't care less about your content) it is very likely your posts will be to a great degree wasted on a useless audience. In addition to this they have NOT installed a mechanism through which I can engage with my entire audience. I understand that people don't want their timeline spammed. At the same time, they liked the page. If they liked Tesla it is likely they want to hear from Tesla. Facebook provides no mechanism through which you can do this (message your entire audience) unless you pay them for every single point of contact on every single message/post. This, to me, is fundamentally wrong. They don't own my customers. I do. A fan page or group owner should not have to pay FB over and over again to reach his or her audience. It's a scam. In sharp contrast to this, if I spend the same $100K advertising on Google and get my customers/likes/fans to register on my site I can then build a real relationship with them over time. If they don't like what I am saying they can opt out of the emails they receive. If they do, they keep receiving them. And I don't have to spend tens of thousands of dollars every time I need to communicate with them. A more concrete example is a product we tested on FB. It was aimed at medical professionals. With custom audiences you'd think you can reach a good number of your prospects. That isn't necessarily true. The heuristics they use can deliver anyone who, for example, likes pages with medical information. I am being simplistic. I think you get the point. Still, a good argument could be made that one could do worst outside FB. In terms of targeting a narrow audience this might mostly be true depending on the approach taken. Reaching medical professionals on FB is expensive. Likes can easily cost you more than a dollar when all is said and done. And, generally speaking, there isn't a 1 to 1 relationship between ad spend and actions or page likes. So, you spend $100K to reach medical professionals and you might get 1,000 to 5,000 legitimate likes. In other words, you could actually spend a very real $20 to $100 per prospect. And then you have to spend money again every single time you need to reach them. We all know that a marketing message must be received multiple times before action is produced. You could very easily spend another $10 per prospect before you get a conversion. That's why I say this is a huge scam. If I spent $20 to $100 per effective member to a page or group, from that point forward they should be my members, prospects or customers, not Facebook's. And extorting money out of me in order to reach all of them with every post is just plain wrong. The same scenario plays out far better if you avoid FB, use Google and other channels to reach your desired audience and bring people to your own environment for engagement. We've tested this with various audiences and also had plenty of prior experience before advertising on FB. We can consistently produce better results out of direct engagement with an email list as small as 200 people than by spending almost any amount of money on FB. This is why I don't understand when I see TV ads where companies are subverting their brands to FB by telling people to go to their FB page rather than their own landing pages. They spend millions of dollars creating and airing these ads and then they have to go back and shovel more money at FB to reach the very people they spent so much money to target. That's just insane. More people doesn't mean better results. In the end I think it is about a basic business formula: Is there a more cost-effective and higher-converting approach to reaching customers and building relationships with them than using FB ads? At the moment my answer to that question is: Yes. Absolutely. I could be wrong but my experiences so far tell me otherwise.
- userbinator 12y agoThis reminds me of http://dhowe.github.io/AdNauseam/ http://dhowe.github.io/AdNauseam/ , which was on HN recently: https://news.ycombinator.com/item?id=8515398 https://news.ycombinator.com/item?id=8515398
- walterbell 12y agoWhiteOps is working on ad-fraud detection: http://blogs.wsj.com/cmo/tag/whiteops/ http://blogs.wsj.com/cmo/tag/whiteops/ & http://www.whiteops.com/ http://www.whiteops.com/
- SixSigma 12y ago> Do you hate how easy it is for your computer to become slow and riddled with malware? No, I can't say I do. If such crime is so rampant, one would think it was ripe for law enforcement to do more about it. I wonder if reporting it to the police every time one visits a relative and removes crap from their computer would start a disruption.
- andybak 12y agoDo pop back here and let us know how that works out for you.
- SixSigma 12y agoExcellent sarcasm. Did mummy help you?
- GFischer 12y agoI went to the local electronic crimes branch of law enforcement once, when my Gmail account was broken into. They basically asked me if I was reporting large-scale financial theft, human traficking or child porn. If it wasn't one of those, they didn't have the manpower or will to pursue it.
- aruggirello 12y agoI have an (unfinished) anti-bot PHP project that does customizable automated bot tests and blocking (both via 403 forbidden, and iptables). A custom version of this script is running at one e-commerce site since 2012. The 403 page has a "ticket ID" so the (very few) real users caught can still contact the website admins to be unblocked. The (almost feature-complete) version running at the site above was able to automatically identify and successfully block a number of bots found scraping (or, attempting to scrape) the site since 2012. Though it wasn't an easy task, it has been quite successful at blocking most of them since their second (often first) request. In fact since the beginning, and largely upto now, most bots and botnets can still be easily spotted. Some by their user-agent, some falling into bot traps ("honeypots") which are invisible to regular users, some failing to load images (something that a regular user wouldn't do), some crawling the site at an unreasonable pace... most usually don't provide an http referer, some even appear to have a number of cookies set for your domain. It's not always easy to spot them though - some botnets can only be undone by blocking their full IP subnet, or permanently banning their (sub)domains, like .cn, .ua or .br (if you don't have legitimate traffic from those domains). Unfortunately, I haven't had enough spare time to complete my project (which tests all of the above conditions to spot bots, but is not ready for a release), but I could if there is some market - or I could sell the project if somebody is interested.
- jacquesm 12y agoIt's not hard to make a 70% effective bot blocker. It's extremely hard to make a 95%+ effective bot blocker without having a large number of false positives. It is (probably) impossible to make a 99%+ effective bot blocker without having a very large number of false positives. I've been working on this (and abusive user blocking) for a long time now and I'm roughly where you are, I have a product, it could be launched but I'm very wary of launching a half baked product that would eventually end up being disabled. I've had the help of a friendly corporate entity that gave me millions of datapoints to play with allowing me to look into their kitchen both on the support end and on the incoming traffic end as well as verified abusers. This identified various fraud such as ad click fraud, affiliate scams, repeated harassment by the same users under different names, spam of all kinds and so on. It's been working there for more than 4 months on a fairly high level of confidence. And yet, I'm still wary of rolling it out on a larger scale.