6 ms·

As this is a repost (no offense to OP), I'm going to repost my original comment (thread here: https://news.ycombinator.com/item?id=21468505 https://news.ycombin

As this is a repost (no offense to OP), I'm going to repost my original comment (thread here: https://news.ycombinator.com/item?id=21468505 https://news.ycombinator.com/item?id=21468505) about this which is still applicable: Reliably someone comes along every few months to question digital ads. I always come back to analyses of incrementality as the real proof. Take an audience of X people. Divide them in two. Show ads to your test group, don't show to control. Watch your business grow and gauge the lift between the two audiences. The companies that know how to advertise at scale do this constantly and can gauge the real effect of their ad dollars. Facebook, Google and others make these tests possible in their platforms, while other software suites such as Impact Altitude and VisualIQ allow you to do this kind of analysis and testing as well. In the end, most of it proves out to be incremental. There are notable exceptions of course, but when are there not?

0xDEEPFAC 6y ago
The thing is I am not sure a large percentage of advertisers are aware of such things and adjust their expectations. Facebook and their ilk encourage you to believe their numbers even though they are mostly phony - I mean how many people do you know that haven't block ads that actually click on them without them making their ui so confusing that you do it by accident? The market is overhyped
- jakear 6y ago
  I block ads on desktop, but on things like mobile Instagram, there’s not much you can do. To some extent I’ve learned to stop worrying and love the tracking, and I will with some regularity tap through to story ads I see with genuine interest in the product (generally outdoorsy things). I’ve not bought anything yet because the websites have failed to convince me the product isn’t junk, but that’s their problem — Facebook has successfully completed their side of the deal (get eyes on the website).
  - throwaway8941 6y ago
    Have you tried DNS-based ad blocking like pi-hole or https://nextdns.io/ https://nextdns.io/ ?
    - jakear 6y ago
      I’m likely different from many hn readers, but the ads really don’t bother me all that much, and fussing with technology bothers me quite a lot. I like my personal devices to Just Work (tm), and configuring dns/setting up raspberry pi’s/etc is the antithesis of that. If I have free time I’ll spend it in the shop, kitchen, or outside — absolutely not fussing with technology.
- kposehn 6y ago
  > The thing is I am not sure a large percentage of advertisers are aware of such things and adjust their expectations. True. Advertiser education is a big challenge for platforms, agencies and marketing employees. Every time I go to a new company I have to educate people on the nuts and bolts so they understand how it works. By and large, they get it and the value gets proven in no uncertain terms. > Facebook and their ilk encourage you to believe their numbers even though they are mostly phony The tests I refer to get around that. If you spend $X on ads to one group and they come in the door and spend more money, how is that phony? And where are there actual ads stats on a campaign a company has purchased that are phony by FB? They've had issues around their marketing of ad statistics externally, but after well into the mid nine figures spent there in my career I've never seen a single phony ad stat, only mistakes we made ourselves that inflated/compressed our numbers. > I mean how many people do you know that haven't block ads that actually click on them without them making their ui so confusing that you do it by accident? You're thinking of content ads on low quality publisher that are trying to stuff as many ways to monetize on their site at the cost of customer experience. Those are total crap most of the time and, yes, often are not incremental. And of course, there's a population of people with ad blockers. But they still search, they still click affiliate links and they are still affected by larger brand advertising in the end. It is pervasive. > The market is overhyped Definitely not going to disagree there!
- nerdponx 6y ago
  You don't have to actually click on the ad anymore. There was a period of time when I was seeing a lot of ads for Bonobos pants. If I bought the pants as a result, I'd be shocked if my record as "Bonobo pants buyer" was not linked to my record as "Bonobo ads viewer". This is one of the purposes of all the tracking.
aguyfromnb 6y ago
>Take an audience of X people. Divide them in two. Show ads to your test group, don't show to control. Watch your business grow and gauge the lift between the two audiences. How are you measuring the group you don't show ads to? This is exactly the sort of experiment that the economists in this article (both independent and from Facebook) did, and measured substantial selection effects. The fact that there are businesses built on businesses built on businesses that are, essentially, allowing people to compete for tiny ad space on specific keywords demonstrates that there is too much hype and obscurity in the space.
- kposehn 6y ago
  In this case, you create a test and control of two audiences using a variety of techniques: 1. Do 1st party -> 3rd party data matching with a partner like LiveRamp or Oracle (formerly DataLogix), where you effectively take consumer data you have gathered and then match it against a targeting set. Then you select one group as test and one as control. Depending on the size and characteristics of your dataset, you often will work to randomize that audience across different traits (age, gender, HHI, etc) to ensure that you get two groups that are effectively the same. Your primary purpose here is retargeting/upsales or using data you have purchased to drive new customer acquisition. 2. Do a 1st party data -> 3rd party lookalike with a partner like Facebook or Google, which is then randomized into two groups for test and control through their own system. Pretty straightforward and great for acquisition. Then you run ads to your test group and none to control. Or maybe control sees your current message, and you test a new one with the test group. Or maybe you have more than one test group to gauge the lift on your business. Lots of options. Now, you gauge your results by looking at the fundamental shift in consumer behavior that occurs in your test group vs. control. You're really trying to say "hey, we showed X ads at $Y cost to this group of Z people. Our control group saw 2% become customers, but the test group saw 6% become customers. Thus our incremental lift is the amount over the baseline of control." This is a totally oversimplified way of doing it because the reality is vastly more complex, but that is the gist of it. Many advertisers and agencies are savvy to this methodology, but an even larger number are not. Those that test and iterate this way are the ones who are effective at driving growth. Now, to your second point: > The fact that there are businesses built on businesses built on businesses that are, essentially, allowing people to compete for tiny ad space on specific keywords demonstrates that there is too much hype and obscurity in the space. Don't conflate bottom of funnel high-intent search ads with your general digital ads. In the case of Google, you are bidding on intent at that moment, probably one of the most valuable things you can advertise against. This is why Google prints so much money - that search, such as "used 2020 MacBook Pro for sale" is about the highest indicator of someone who intends to buy you can get outside of them banging on your door saying "take my money!!" This is about the most incremental advertising you can ever get for 99% of companies that are not Amazon. Amazon doesn't care because most of the time people go search on Amazon first, but they're still willing to advertise for the 0.001% of their customers who aren't starting with them.
jariel 6y ago
The fundamental problem here is that very few sales are 'cold and direct'. Very seldom does the funnel work like: 'Target sees and ad, buys the product or not'. All ads ultimately have a lingering buy effect, and all ads are essentially also a form of brand advertising. When they see your 'test ad' the first time, it makes them more or less likely to buy your product at some future date, the next time they are in-store or whatever. For some kinds of things what you describe works, but for other kinds of things, and for bigger companies, not so much, or rather, it's harder to do.
- jlbnjmn 6y ago
  The lingering impact of awareness is fascinating and underappreciated. It takes advantage of the familiarity effect. It would be fun to measure. Buy the cheapest add to get your brand seen by as many people as possible in a market, and then spend on the highest intent keywords. See what it does to total ROI.
wdr1 6y ago
> Take an audience of X people. Divide them in two. Show ads to your test group, don't show to control. That works for the simplest of advertiser campaigns, but doesn't scale. How do you break down your audience across platforms? I.e., how do you divide your audience in half across your ads buys on Google & Facebook, to make sure someone excluded from your ads on Google is excluded on Facebook? And that's not even the hard problem of how you do it across digital, television, billboards, magazines & so on. Even if you could get that data, if your campaign is flighted, you don't have enough time to act. If you have a big movie opening or product launch, you can't wait to see the initial response & then optimize. These approaches can work for someone taking advantage of the latest Instagram Dropship scam, but they're going to work for where the bulk of advertising dollars are (big brands, movies, cars, consumables, etc.)
- kposehn 6y ago
  > That works for the simplest of advertiser campaigns, but doesn't scale. Actually, the examples I'm referring to are at pretty massive scale (>$100mm in spend) all the way down to <$100k. It works across a pretty massive range. >. How do you break down your audience across platforms? I.e., how do you divide your audience in half across your ads buys on Google & Facebook, to make sure someone excluded from your ads on Google is excluded on Facebook? This is a challenge for sure and it depends on how you do it, and its also where vendors like LiveRamp and Oracle do so well. As they have very high quality data sets that work across platforms they can do these sorts of tests and exclusions. However! That isn't actually necessary. Typically when you do a test, you will isolate the test to certain ranges and hold other campaigns constant. As long as you have enough data to create a solid predictive model you can pretty easily isolate your effects between different channels. > And that's not even the hard problem of how you do it across digital, television, billboards, magazines & so on. See my above answer :) > Even if you could get that data, if your campaign is flighted, you don't have enough time to act. If you have a big movie opening or product launch, you can't wait to see the initial response & then optimize. Now this is a challenge for sure. For big launches, its much harder to create this kind of testing during the campaign. This is why testing ahead of time is important and for big launches media buyers are running off what they already know about a channel vs. testing during. > These approaches can work for someone taking advantage of the latest Instagram Dropship scam, but they're going to work for where the bulk of advertising dollars are (big brands, movies, cars, consumables, etc.) Completely disagree. If anything, this doesn't work at all for small scale advertisers unless they spend real $'s on their campaigns (>$30k/month at least). Technically you could do a test like this at small scale, but unless your CPA is super low or you get a metric ton of clicks and impressions, you won't be able to get a key learning from it.

0xDEEPFAC 6y ago

The thing is I am not sure a large percentage of advertisers are aware of such things and adjust their expectations. Facebook and their ilk encourage you to believe their numbers even though they are mostly phony - I mean how many people do you know that haven't block ads that actually click on them without them making their ui so confusing that you do it by accident? The market is overhyped

jakear 6y ago
I block ads on desktop, but on things like mobile Instagram, there’s not much you can do. To some extent I’ve learned to stop worrying and love the tracking, and I will with some regularity tap through to story ads I see with genuine interest in the product (generally outdoorsy things). I’ve not bought anything yet because the websites have failed to convince me the product isn’t junk, but that’s their problem — Facebook has successfully completed their side of the deal (get eyes on the website).
- throwaway8941 6y ago
  Have you tried DNS-based ad blocking like pi-hole or https://nextdns.io/ https://nextdns.io/ ?
  - jakear 6y ago
    I’m likely different from many hn readers, but the ads really don’t bother me all that much, and fussing with technology bothers me quite a lot. I like my personal devices to Just Work (tm), and configuring dns/setting up raspberry pi’s/etc is the antithesis of that. If I have free time I’ll spend it in the shop, kitchen, or outside — absolutely not fussing with technology.
kposehn 6y ago
> The thing is I am not sure a large percentage of advertisers are aware of such things and adjust their expectations. True. Advertiser education is a big challenge for platforms, agencies and marketing employees. Every time I go to a new company I have to educate people on the nuts and bolts so they understand how it works. By and large, they get it and the value gets proven in no uncertain terms. > Facebook and their ilk encourage you to believe their numbers even though they are mostly phony The tests I refer to get around that. If you spend $X on ads to one group and they come in the door and spend more money, how is that phony? And where are there actual ads stats on a campaign a company has purchased that are phony by FB? They've had issues around their marketing of ad statistics externally, but after well into the mid nine figures spent there in my career I've never seen a single phony ad stat, only mistakes we made ourselves that inflated/compressed our numbers. > I mean how many people do you know that haven't block ads that actually click on them without them making their ui so confusing that you do it by accident? You're thinking of content ads on low quality publisher that are trying to stuff as many ways to monetize on their site at the cost of customer experience. Those are total crap most of the time and, yes, often are not incremental. And of course, there's a population of people with ad blockers. But they still search, they still click affiliate links and they are still affected by larger brand advertising in the end. It is pervasive. > The market is overhyped Definitely not going to disagree there!
nerdponx 6y ago
You don't have to actually click on the ad anymore. There was a period of time when I was seeing a lot of ads for Bonobos pants. If I bought the pants as a result, I'd be shocked if my record as "Bonobo pants buyer" was not linked to my record as "Bonobo ads viewer". This is one of the purposes of all the tracking.

aguyfromnb 6y ago

>Take an audience of X people. Divide them in two. Show ads to your test group, don't show to control. Watch your business grow and gauge the lift between the two audiences. How are you measuring the group you don't show ads to? This is exactly the sort of experiment that the economists in this article (both independent and from Facebook) did, and measured substantial selection effects. The fact that there are businesses built on businesses built on businesses that are, essentially, allowing people to compete for tiny ad space on specific keywords demonstrates that there is too much hype and obscurity in the space.

kposehn 6y ago
In this case, you create a test and control of two audiences using a variety of techniques: 1. Do 1st party -> 3rd party data matching with a partner like LiveRamp or Oracle (formerly DataLogix), where you effectively take consumer data you have gathered and then match it against a targeting set. Then you select one group as test and one as control. Depending on the size and characteristics of your dataset, you often will work to randomize that audience across different traits (age, gender, HHI, etc) to ensure that you get two groups that are effectively the same. Your primary purpose here is retargeting/upsales or using data you have purchased to drive new customer acquisition. 2. Do a 1st party data -> 3rd party lookalike with a partner like Facebook or Google, which is then randomized into two groups for test and control through their own system. Pretty straightforward and great for acquisition. Then you run ads to your test group and none to control. Or maybe control sees your current message, and you test a new one with the test group. Or maybe you have more than one test group to gauge the lift on your business. Lots of options. Now, you gauge your results by looking at the fundamental shift in consumer behavior that occurs in your test group vs. control. You're really trying to say "hey, we showed X ads at $Y cost to this group of Z people. Our control group saw 2% become customers, but the test group saw 6% become customers. Thus our incremental lift is the amount over the baseline of control." This is a totally oversimplified way of doing it because the reality is vastly more complex, but that is the gist of it. Many advertisers and agencies are savvy to this methodology, but an even larger number are not. Those that test and iterate this way are the ones who are effective at driving growth. Now, to your second point: > The fact that there are businesses built on businesses built on businesses that are, essentially, allowing people to compete for tiny ad space on specific keywords demonstrates that there is too much hype and obscurity in the space. Don't conflate bottom of funnel high-intent search ads with your general digital ads. In the case of Google, you are bidding on intent at that moment, probably one of the most valuable things you can advertise against. This is why Google prints so much money - that search, such as "used 2020 MacBook Pro for sale" is about the highest indicator of someone who intends to buy you can get outside of them banging on your door saying "take my money!!" This is about the most incremental advertising you can ever get for 99% of companies that are not Amazon. Amazon doesn't care because most of the time people go search on Amazon first, but they're still willing to advertise for the 0.001% of their customers who aren't starting with them.

jariel 6y ago

The fundamental problem here is that very few sales are 'cold and direct'. Very seldom does the funnel work like: 'Target sees and ad, buys the product or not'. All ads ultimately have a lingering buy effect, and all ads are essentially also a form of brand advertising. When they see your 'test ad' the first time, it makes them more or less likely to buy your product at some future date, the next time they are in-store or whatever. For some kinds of things what you describe works, but for other kinds of things, and for bigger companies, not so much, or rather, it's harder to do.

jlbnjmn 6y ago
The lingering impact of awareness is fascinating and underappreciated. It takes advantage of the familiarity effect. It would be fun to measure. Buy the cheapest add to get your brand seen by as many people as possible in a market, and then spend on the highest intent keywords. See what it does to total ROI.

wdr1 6y ago

> Take an audience of X people. Divide them in two. Show ads to your test group, don't show to control. That works for the simplest of advertiser campaigns, but doesn't scale. How do you break down your audience across platforms? I.e., how do you divide your audience in half across your ads buys on Google & Facebook, to make sure someone excluded from your ads on Google is excluded on Facebook? And that's not even the hard problem of how you do it across digital, television, billboards, magazines & so on. Even if you could get that data, if your campaign is flighted, you don't have enough time to act. If you have a big movie opening or product launch, you can't wait to see the initial response & then optimize. These approaches can work for someone taking advantage of the latest Instagram Dropship scam, but they're going to work for where the bulk of advertising dollars are (big brands, movies, cars, consumables, etc.)

kposehn 6y ago
> That works for the simplest of advertiser campaigns, but doesn't scale. Actually, the examples I'm referring to are at pretty massive scale (>$100mm in spend) all the way down to <$100k. It works across a pretty massive range. >. How do you break down your audience across platforms? I.e., how do you divide your audience in half across your ads buys on Google & Facebook, to make sure someone excluded from your ads on Google is excluded on Facebook? This is a challenge for sure and it depends on how you do it, and its also where vendors like LiveRamp and Oracle do so well. As they have very high quality data sets that work across platforms they can do these sorts of tests and exclusions. However! That isn't actually necessary. Typically when you do a test, you will isolate the test to certain ranges and hold other campaigns constant. As long as you have enough data to create a solid predictive model you can pretty easily isolate your effects between different channels. > And that's not even the hard problem of how you do it across digital, television, billboards, magazines & so on. See my above answer :) > Even if you could get that data, if your campaign is flighted, you don't have enough time to act. If you have a big movie opening or product launch, you can't wait to see the initial response & then optimize. Now this is a challenge for sure. For big launches, its much harder to create this kind of testing during the campaign. This is why testing ahead of time is important and for big launches media buyers are running off what they already know about a channel vs. testing during. > These approaches can work for someone taking advantage of the latest Instagram Dropship scam, but they're going to work for where the bulk of advertising dollars are (big brands, movies, cars, consumables, etc.) Completely disagree. If anything, this doesn't work at all for small scale advertisers unless they spend real $'s on their campaigns (>$30k/month at least). Technically you could do a test like this at small scale, but unless your CPA is super low or you get a metric ton of clicks and impressions, you won't be able to get a key learning from it.