11 ms·

You can A/B test ads pretty easily, and this is quite common. With some degree of statistical certainty, you can tell how one ad performs to another. You don't

You can A/B test ads pretty easily, and this is quite common. With some degree of statistical certainty, you can tell how one ad performs to another. You don't have control over your SEO results as well - but you can also measure against SEO traffic with a high degree of certainty. All big companies do this. Sure, you're never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it. But that honestly doesn't seem that important compared to the other metrics - which most functioning large companies have decent data on. You're also never going to know how many of your customers ate Green Eggs and Ham for breakfast. It's irrelevant. You have decent insight into your direct-online ROAS, and that's unique to direct online advertising, and it's important!! Yes, you're never going to know if that million dollars you invested in online ads was the best use of that million dollars. But that's not much different than building a new factory, either. Edit: Specifically to Uber's case - they did NOT turn off 66% of ALL ads randomly to no adverse effect (implying that all ads are worthless). They DISCOVERED that a certain type of ad (paying for installs on dubious ad networks) was mostly fraud. After turning off 100% of this type of ad - they found no adverse effect. This is a fail on their analytics team. They should've been measuring this type of ad better - especially given how big a portion of the total spend it was - and had insight into something not being right. They should have been able to do this - and if they couldn't, because the network somehow didn't give them enough data to do it, they probably shouldn't have been spending this much money for exactly these reasons!

isoprophlex 6y ago
Did you actually read the linked tweets?! Uber turns off a shitload of ad spending, nothing bad happens to new user acquisitions. I'd say there are about a million better uses of a million dollars than just pissing them away on a scam.
- thrav 6y ago
  There’s an interesting question there about when this form of advertising becomes obsolete. Uber is in a very different position than many other companies. Anyone who browses the internet with regularity is already aware of their existence and probably just needs the right set of circumstances to come together to make Uber useful to them. I suspect the results would be different if Uber were earlier in their adoption curve, but maybe that’s not true either. Maybe they’d be ignored for different reasons at that time.
  - wasdfff 6y ago
    Likewise, I probably haven’t seen an ad for coca cola or kleenex in a while. Once a brand is ubiquitous to the point where soda becomes coke and tissues become kleenex in the lexicon, it feels like ad spend is wasted.
    - jhawk28 6y ago
      Brand advertising is different. It is there to remind people about the brand. Companies do it because they have the analytics that show that it works.
      - Retric 6y ago
        They mostly do it from inertia, how well it works is a separate question. Historically you can find many examples of hugely successful advertising campaigns, it’s much harder to quantify the negative.
      - z3t4 6y ago
        I think both Uber and CocaCola has a lot of work to do until I like any of them. I like CocaCola as a product though. Never tried Uber. For example I really like some Nestle products, but try to avoid them because of their bad company/brand reputation, even if their product is both cheaper and tastier I buy from a competitor instead. Whatever a company stands for and how they act will affect their brand.
    - DanBC 6y ago
      Coca Cola spends billions on ads. 8 million views: https://www.youtube.com/watch?v=yg4Mq5EAEzw https://www.youtube.com/watch?v=yg4Mq5EAEzw
      - coldtea 6y ago
        >8 million views So, less than a viral cat video?
    - ajsnigrutin 6y ago
- walshemj 6y ago
  Paying for crappy app installs on shady sites is not really advertising. What they should have done was a proper hyperlocal SEO campaign like Firestone Gieco, and Mc Donalds do.
- m00x 6y ago
  There's a huge difference between a company that is widely known for being the app for ride hailing and a small company trying to get eyes on their new product.
- coldtea 6y ago
  Well, Uber is known to everybody by now. To the effect that "to Uber" is even kind of a verb/noun. Also people either need a hired transportation or not. If they don't have a car or don't want to mess with the traffic and need to go somewhere, it's either Uber or Taxi usually. It's not like Coca Cola, which is well known, but people could do without it (unless addicted), so needs to constantly nag people. And it's not like some new product, which without advertising nobody would even know it existed. In fact most of Uber's existance its operation has been 100% advertising (spending VS money to offer cheap rides and expand and gather "eyeballs" and "customers" without a profit). In my book, customer acquisition without profit is another name for advertising. So it doesn't sound strange that it could do without advertising today. But what if there were 3-4 strong players in the same, each eating in Uber's market share? You'll see how fast they'd found advertising indispensable again...
- ashtonkem 6y ago
  Literally lighting them on fire in a cold office to help provide some BTUs would’ve been a better use of that money, since at least that would offset spending somewhere else.
azornathogron 6y ago
> All big companies do this. Except Uber, apparently? Or would the method you're talking about not have discovered that something fishy was going on? (I don't work with ads so I don't know the limitations of the type of experiment that you're talking about)
- onlyrealcuzzo 6y ago
  It would work for some of Uber's ads. From the article, it looks like they were frauded mostly by in-app ads. And they were paying for installs - not actual trips. So, no, Uber's advertising here is a little different than (I think) the majority of companies. They are mostly paying for installs rather than sales / conversions. A lot of newer "app" companies could be in similar situations. Though, honestly, this seems like a massive fail on their analytics team for not figuring this out earlier. They should have been able to see that all of these "installs" from certain advertisers were not leading to trips. In fact, it says they turned off 66% of ads. They didn't randomly turn of 66% of ALL ads. They turned off this TYPE of ad, which they failed to earlier recognize was ineffective. Step 1) assume your ads won't work. Step 2) have enough analytics / logging in place to convince yourself the ads do work. Step 3) if they don't work, turn them off. Looks like they skipped step 2 - which honestly, is not uncommon for a fast growing business - even if they are huge and already make a lot of money. What they found isn't even what people are discussing. They found that certain networks they were buying ads from were almost 100% fraud (which is pretty well known). Instead, people here seem to be discussing that most online advertising is fraud, and/or that there's no way to prove it's effective. That is absurd.
  - lincolnq 6y ago
    Hang on though, in the linked thread they said that the fraud was incurred at the point of real users signing up for uber -- when they type uber into the app store search (e.g. an organic install), the ad network fraudulently takes credit for it at that point. So "enough analytics/logging" would not do the trick here - I think they would have noticed if a certain type of ad wasn't leading to trips, as you are pointing at.
LoSboccacc 6y ago
> You can A/B test ads pretty easily A/B testing ads is a complex matter. you can A/B traffic and conversion easily, but a lot of established companies with fierce competition fight for mind share, not direct conversion; for that, you have both awareness effects (user won't forget about coca cola if they don't run ads for a month, and an ad that doesn't directly convert but increase awareness still has value) and coverage synergies (the number of repetitions in a day will increase coverage non linearly and the amount of channel repetitions will increase awareness more than a single channel view, even if it doesn't convert immediately)
oli5679 6y ago
> Sure, you're never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it. This is typically an order of magnitude less than the attribution figures stated by digital marketing experts. A/B test is the right method to use, but the crucial thing is you need an earmarked population to see zero adds over your attribution window, since what you care about is impact on incremental sales, rather than incremental click likelihood. There is a long econometrics literature on this and it is not a fussy technicality, the figures typically differ by 10x +.
- gregoryl 6y ago
  Instagram definitely does this. I'm part of the magic cohort who never see any ads :)
adrusi 6y ago
It can happen that an almost certainly bogus scientific field persists for decades using state of the art research methods and no bad faith on the part of the researchers (any bias they introduce is probably not conscious). https://slatestarcodex.com/2014/04/28/the-control-group-is-out-of-control/ https://slatestarcodex.com/2014/04/28/the-control-group-is-o... A/B testing is... not a state of the art scientific research technique. Moreover the companies that provide the tools to do A/B testing are the same companies that sell you access to advertising space. I'm not saying that it's a common practice to defraud A/B tests, I'm saying that the fact that adtech companies have chosen to enable that research methodology out of the set of all methodologies they could offer suggests that we should expect, before seeing the results of any A/B trial, that the results will tend to favor the adtech narrative. I don't think that people in the adtech space believe they're selling a bogus product, but I do think they wouldn't want to know if they were — they have a good thing going. If you're employed in adtech you're mostly fine, skills transfer. If you're invested in adtech, do what you can to diversify away. Advertising is overvalued to some extent and that bubble will burst at some point or other. The question is just how much actual value advertising provides, how much will remain when the bubble bursts.
nabla9 6y ago
>but you can also measure against SEO traffic with a high degree of certainty. None of these benchmarks distinguish between the selection effect (clicks, purchases and downloads that are happening anyway) and the advertising effect (clicks, purchases and downloads that would not have happened without ads). You can fix this by dividing the target group into two random cohorts in advance: one group sees the ad, the other does not. Designing the experiment thus excludes the effects of selection. When you do this experiment correctly, you find out that ads have low effect or are not cost effective (as eBay discovered).
maxerickson 6y ago
So how much ad spending is going to 'dubious ad networks'? Are they easy to categorize? Or is it a big secret what's dubious and what isn't?
nojito 6y ago
This is all discussed in the podcast. There’s no revenue change even after testing (when you tack on costs of ad delivery).
ransom1538 6y ago
"You can A/B test ads pretty easily, and this is quite common. With some degree of statistical certainty, you can tell how one ad performs to another." Nope. You need a unique id for an AB test AND apple wont give this to you. No company is going to give you a click/tap id. Android might. This is so inaccurate I don't know where to start. Apple has turned off all unique ids that work across an AB test [1] cross platform. You would need on the ground mobile ad experience to know this. https://www.invoca.com/blog/what-is-idfa-and-why-apple-killed-it-everything-marketers-need-to-know https://www.invoca.com/blog/what-is-idfa-and-why-apple-kille...
- jameshush 6y ago
  For native Android you 100% have access to an Android "device id". I set this up on an Android app we delivered. There's even companies that pay other Android apps to host their code inside the app, co-related that ID to other data available to the web (e.g. IP address, installed fonts, etc) to provide a very good guess of a user's device id via the web. Apple had something similar, but you're right they're starting to turn it off. In my past life in Adtech we noticed the click through rate of users with Android user agents were a lot higher on certain campaigns than iOS users for this reason.
Reebz 6y ago
Across this and the other[0] highly active digital ads post, there's 3 interesting forces at play: a) technologists screaming "ads are literally the worst societal cost, like ever" who don't want to understand the industry b) advertising folks taking up arms to defend their shamanistic, money-printing machines without statistics (Reason No. 7 will SHOCK you) c) A tiny, tiny group of Mandalorian-like voices who have a necessary statistics AND industry understanding who are being drowned out (this is in jest, but I double-dare you to say it isn't at least directionally accurate) Other commenters have mentioned it throughout this whole comment thread as "incrementality tests". Amongst other approaches, this is the way. Freakonomics severely lack the industry understanding. Listening to the podcast was like hearing how HTML is a programming language from the kid in week 2 at code camp. Then too the article, all the issues with Uber was just doing a bad job of managing their ad spend and they can fall into group B, noted above. The baseline of this work is a control group who see no ads and then you build your tests from there that factor in channels, cohorting, and other components to get a statistically significant outcome. Yes, this will get more challenging with upcoming privacy changes (IDFA removal, et al). However, the last 10 years this wasn't a problem and I'm sure the corporations in the identity resolution business will hand-shake on a bunch of 2nd party data deals that just move the deals done in broad daylight around identity tracking and audience creation to the alley. Further, any advertising that is tied to an already known customer is able to be backed into at an audience level with login and cookie data. Even Pi-hole users may not be exempt here. To finish with some constructive advice: 1) Advertising is not a synonym for marketing. We're only talking about advertising in both HN threads. 2) Every industry has high and low quality. Pareto's principle should be aggressively applied to where one spends their budget in the cesspool that is the Internet. 3) If you're ever seeking an agency to provide advertising services and they don't have a qualified data science or statistics leader (10+ years work experience, degree in Stats/Math/Econ, an MBA, or similar), run. Run from those shiny-shoes gurus. Channel your inner Usain Bolt and run. 0. https://news.ycombinator.com/item?id=25620707 https://news.ycombinator.com/item?id=25620707
dillondoyle 6y ago
You can definitely measure 'lift' - though it's far easier with online only sales. For sure it's harder for say Gucci to measure incremental cross-platform lift but any good large brand spends big money trying to parse it out One example way to get some value measurement would be Facebook's powerful tools. Beyond a basic 1:1 audience 50/50 exposure test, Facebook also lets you upload offline sales (or use their pretty effective pixel for real time online sales) and then feed into their system. Using the data your purchase data FB matches to ads delivered and you (with PII but also purchase value and frequency). Then the advertiser can define lookback/conversion definitions to get a measure of incremental lift. E.g. those who clicked ad bought __ right away, those who viewed ad within 1, 7 or 28 days bought __. For what it's worth the adtech forums I participate on view Uber as incompetent idiots on this issue (which has been reported on like last year) - all they had to do was measure beyond the initial install for say did these installs actually pay to ride... Install fraud from SSPs and resellers is huge, doing this basic measurement and quality control is the reason Facebook and Apple have huge and growing app install business. Though TBD on Apple fucking over Facebook and keeping the attribution for themselves alone with their new privacy features
st1x7 6y ago
> Sure, you're never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it. But that honestly doesn't seem that important compared to the other metrics... Wait, why wouldn't that be important? It seems like the most important question since it asks whether advertising has any significant impact at all.
- megablast 6y ago
  Does advertising create an increase in buying your product. That’s the most important question.
  - monkeybutton 6y ago
    At least with ecommerce one can track users that have seen and/or clicked on ads with what they have purchased and return on ad spend can be calculated. But yes, the question still is: does it raise intent or did the user already want the thing, googled it, and clicked on the first result (which is your ad, above the organic results that also lead to your online store)?
    - dlp211 6y ago
      The fundamental problem with this is that if I bought your product, it's because I was looking for your product after doing a decent amount of research which meant I was on a bunch of retargeting lists and you've wasted money on me. I have never to my recollection purchased anything because of an ad. If I have clicked through an ad, it was simply because it was the quickest way to get to the website/product. There is a role for advertising because Google polluted their organic search results with Ads making it harder for you to find what you want and forcing you to compete for the top Ad spot. I realize that I can only speak for myself, but I have never seen any evidence that advertising works and I've worked in ad tech.
      - ipsum2 6y ago
        It's pretty clear to see direct causation when using ads to promote your product. Promote your product using ads for 1 month, then turn off ads and see how many sales you get. For a majority of e-commerce, you'll see a substantial drop.

isoprophlex 6y ago

Did you actually read the linked tweets?! Uber turns off a shitload of ad spending, nothing bad happens to new user acquisitions. I'd say there are about a million better uses of a million dollars than just pissing them away on a scam.

thrav 6y ago
There’s an interesting question there about when this form of advertising becomes obsolete. Uber is in a very different position than many other companies. Anyone who browses the internet with regularity is already aware of their existence and probably just needs the right set of circumstances to come together to make Uber useful to them. I suspect the results would be different if Uber were earlier in their adoption curve, but maybe that’s not true either. Maybe they’d be ignored for different reasons at that time.
- wasdfff 6y ago
  Likewise, I probably haven’t seen an ad for coca cola or kleenex in a while. Once a brand is ubiquitous to the point where soda becomes coke and tissues become kleenex in the lexicon, it feels like ad spend is wasted.
  - jhawk28 6y ago
    Brand advertising is different. It is there to remind people about the brand. Companies do it because they have the analytics that show that it works.
    - Retric 6y ago
      They mostly do it from inertia, how well it works is a separate question. Historically you can find many examples of hugely successful advertising campaigns, it’s much harder to quantify the negative.
    - z3t4 6y ago
      I think both Uber and CocaCola has a lot of work to do until I like any of them. I like CocaCola as a product though. Never tried Uber. For example I really like some Nestle products, but try to avoid them because of their bad company/brand reputation, even if their product is both cheaper and tastier I buy from a competitor instead. Whatever a company stands for and how they act will affect their brand.
  - DanBC 6y ago
    Coca Cola spends billions on ads. 8 million views: https://www.youtube.com/watch?v=yg4Mq5EAEzw https://www.youtube.com/watch?v=yg4Mq5EAEzw
    - coldtea 6y ago
      >8 million views So, less than a viral cat video?
  - ajsnigrutin 6y ago
walshemj 6y ago
Paying for crappy app installs on shady sites is not really advertising. What they should have done was a proper hyperlocal SEO campaign like Firestone Gieco, and Mc Donalds do.
m00x 6y ago
There's a huge difference between a company that is widely known for being the app for ride hailing and a small company trying to get eyes on their new product.
coldtea 6y ago
Well, Uber is known to everybody by now. To the effect that "to Uber" is even kind of a verb/noun. Also people either need a hired transportation or not. If they don't have a car or don't want to mess with the traffic and need to go somewhere, it's either Uber or Taxi usually. It's not like Coca Cola, which is well known, but people could do without it (unless addicted), so needs to constantly nag people. And it's not like some new product, which without advertising nobody would even know it existed. In fact most of Uber's existance its operation has been 100% advertising (spending VS money to offer cheap rides and expand and gather "eyeballs" and "customers" without a profit). In my book, customer acquisition without profit is another name for advertising. So it doesn't sound strange that it could do without advertising today. But what if there were 3-4 strong players in the same, each eating in Uber's market share? You'll see how fast they'd found advertising indispensable again...
ashtonkem 6y ago
Literally lighting them on fire in a cold office to help provide some BTUs would’ve been a better use of that money, since at least that would offset spending somewhere else.

azornathogron 6y ago

> All big companies do this. Except Uber, apparently? Or would the method you're talking about not have discovered that something fishy was going on? (I don't work with ads so I don't know the limitations of the type of experiment that you're talking about)

onlyrealcuzzo 6y ago
It would work for some of Uber's ads. From the article, it looks like they were frauded mostly by in-app ads. And they were paying for installs - not actual trips. So, no, Uber's advertising here is a little different than (I think) the majority of companies. They are mostly paying for installs rather than sales / conversions. A lot of newer "app" companies could be in similar situations. Though, honestly, this seems like a massive fail on their analytics team for not figuring this out earlier. They should have been able to see that all of these "installs" from certain advertisers were not leading to trips. In fact, it says they turned off 66% of ads. They didn't randomly turn of 66% of ALL ads. They turned off this TYPE of ad, which they failed to earlier recognize was ineffective. Step 1) assume your ads won't work. Step 2) have enough analytics / logging in place to convince yourself the ads do work. Step 3) if they don't work, turn them off. Looks like they skipped step 2 - which honestly, is not uncommon for a fast growing business - even if they are huge and already make a lot of money. What they found isn't even what people are discussing. They found that certain networks they were buying ads from were almost 100% fraud (which is pretty well known). Instead, people here seem to be discussing that most online advertising is fraud, and/or that there's no way to prove it's effective. That is absurd.
- lincolnq 6y ago
  Hang on though, in the linked thread they said that the fraud was incurred at the point of real users signing up for uber -- when they type uber into the app store search (e.g. an organic install), the ad network fraudulently takes credit for it at that point. So "enough analytics/logging" would not do the trick here - I think they would have noticed if a certain type of ad wasn't leading to trips, as you are pointing at.

LoSboccacc 6y ago

> You can A/B test ads pretty easily A/B testing ads is a complex matter. you can A/B traffic and conversion easily, but a lot of established companies with fierce competition fight for mind share, not direct conversion; for that, you have both awareness effects (user won't forget about coca cola if they don't run ads for a month, and an ad that doesn't directly convert but increase awareness still has value) and coverage synergies (the number of repetitions in a day will increase coverage non linearly and the amount of channel repetitions will increase awareness more than a single channel view, even if it doesn't convert immediately)

oli5679 6y ago

> Sure, you're never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it. This is typically an order of magnitude less than the attribution figures stated by digital marketing experts. A/B test is the right method to use, but the crucial thing is you need an earmarked population to see zero adds over your attribution window, since what you care about is impact on incremental sales, rather than incremental click likelihood. There is a long econometrics literature on this and it is not a fussy technicality, the figures typically differ by 10x +.

gregoryl 6y ago
Instagram definitely does this. I'm part of the magic cohort who never see any ads :)

adrusi 6y ago

It can happen that an almost certainly bogus scientific field persists for decades using state of the art research methods and no bad faith on the part of the researchers (any bias they introduce is probably not conscious). https://slatestarcodex.com/2014/04/28/the-control-group-is-out-of-control/ https://slatestarcodex.com/2014/04/28/the-control-group-is-o... A/B testing is... not a state of the art scientific research technique. Moreover the companies that provide the tools to do A/B testing are the same companies that sell you access to advertising space. I'm not saying that it's a common practice to defraud A/B tests, I'm saying that the fact that adtech companies have chosen to enable that research methodology out of the set of all methodologies they could offer suggests that we should expect, before seeing the results of any A/B trial, that the results will tend to favor the adtech narrative. I don't think that people in the adtech space believe they're selling a bogus product, but I do think they wouldn't want to know if they were — they have a good thing going. If you're employed in adtech you're mostly fine, skills transfer. If you're invested in adtech, do what you can to diversify away. Advertising is overvalued to some extent and that bubble will burst at some point or other. The question is just how much actual value advertising provides, how much will remain when the bubble bursts.

nabla9 6y ago

>but you can also measure against SEO traffic with a high degree of certainty. None of these benchmarks distinguish between the selection effect (clicks, purchases and downloads that are happening anyway) and the advertising effect (clicks, purchases and downloads that would not have happened without ads). You can fix this by dividing the target group into two random cohorts in advance: one group sees the ad, the other does not. Designing the experiment thus excludes the effects of selection. When you do this experiment correctly, you find out that ads have low effect or are not cost effective (as eBay discovered).

maxerickson 6y ago

So how much ad spending is going to 'dubious ad networks'? Are they easy to categorize? Or is it a big secret what's dubious and what isn't?

nojito 6y ago

This is all discussed in the podcast. There’s no revenue change even after testing (when you tack on costs of ad delivery).

ransom1538 6y ago

"You can A/B test ads pretty easily, and this is quite common. With some degree of statistical certainty, you can tell how one ad performs to another." Nope. You need a unique id for an AB test AND apple wont give this to you. No company is going to give you a click/tap id. Android might. This is so inaccurate I don't know where to start. Apple has turned off all unique ids that work across an AB test [1] cross platform. You would need on the ground mobile ad experience to know this. https://www.invoca.com/blog/what-is-idfa-and-why-apple-killed-it-everything-marketers-need-to-know https://www.invoca.com/blog/what-is-idfa-and-why-apple-kille...

jameshush 6y ago
For native Android you 100% have access to an Android "device id". I set this up on an Android app we delivered. There's even companies that pay other Android apps to host their code inside the app, co-related that ID to other data available to the web (e.g. IP address, installed fonts, etc) to provide a very good guess of a user's device id via the web. Apple had something similar, but you're right they're starting to turn it off. In my past life in Adtech we noticed the click through rate of users with Android user agents were a lot higher on certain campaigns than iOS users for this reason.

Reebz 6y ago

Across this and the other[0] highly active digital ads post, there's 3 interesting forces at play: a) technologists screaming "ads are literally the worst societal cost, like ever" who don't want to understand the industry b) advertising folks taking up arms to defend their shamanistic, money-printing machines without statistics (Reason No. 7 will SHOCK you) c) A tiny, tiny group of Mandalorian-like voices who have a necessary statistics AND industry understanding who are being drowned out (this is in jest, but I double-dare you to say it isn't at least directionally accurate) Other commenters have mentioned it throughout this whole comment thread as "incrementality tests". Amongst other approaches, this is the way. Freakonomics severely lack the industry understanding. Listening to the podcast was like hearing how HTML is a programming language from the kid in week 2 at code camp. Then too the article, all the issues with Uber was just doing a bad job of managing their ad spend and they can fall into group B, noted above. The baseline of this work is a control group who see no ads and then you build your tests from there that factor in channels, cohorting, and other components to get a statistically significant outcome. Yes, this will get more challenging with upcoming privacy changes (IDFA removal, et al). However, the last 10 years this wasn't a problem and I'm sure the corporations in the identity resolution business will hand-shake on a bunch of 2nd party data deals that just move the deals done in broad daylight around identity tracking and audience creation to the alley. Further, any advertising that is tied to an already known customer is able to be backed into at an audience level with login and cookie data. Even Pi-hole users may not be exempt here. To finish with some constructive advice: 1) Advertising is not a synonym for marketing. We're only talking about advertising in both HN threads. 2) Every industry has high and low quality. Pareto's principle should be aggressively applied to where one spends their budget in the cesspool that is the Internet. 3) If you're ever seeking an agency to provide advertising services and they don't have a qualified data science or statistics leader (10+ years work experience, degree in Stats/Math/Econ, an MBA, or similar), run. Run from those shiny-shoes gurus. Channel your inner Usain Bolt and run. 0. https://news.ycombinator.com/item?id=25620707 https://news.ycombinator.com/item?id=25620707

dillondoyle 6y ago

You can definitely measure 'lift' - though it's far easier with online only sales. For sure it's harder for say Gucci to measure incremental cross-platform lift but any good large brand spends big money trying to parse it out One example way to get some value measurement would be Facebook's powerful tools. Beyond a basic 1:1 audience 50/50 exposure test, Facebook also lets you upload offline sales (or use their pretty effective pixel for real time online sales) and then feed into their system. Using the data your purchase data FB matches to ads delivered and you (with PII but also purchase value and frequency). Then the advertiser can define lookback/conversion definitions to get a measure of incremental lift. E.g. those who clicked ad bought __ right away, those who viewed ad within 1, 7 or 28 days bought __. For what it's worth the adtech forums I participate on view Uber as incompetent idiots on this issue (which has been reported on like last year) - all they had to do was measure beyond the initial install for say did these installs actually pay to ride... Install fraud from SSPs and resellers is huge, doing this basic measurement and quality control is the reason Facebook and Apple have huge and growing app install business. Though TBD on Apple fucking over Facebook and keeping the attribution for themselves alone with their new privacy features

st1x7 6y ago

> Sure, you're never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it. But that honestly doesn't seem that important compared to the other metrics... Wait, why wouldn't that be important? It seems like the most important question since it asks whether advertising has any significant impact at all.

megablast 6y ago
Does advertising create an increase in buying your product. That’s the most important question.
- monkeybutton 6y ago
  At least with ecommerce one can track users that have seen and/or clicked on ads with what they have purchased and return on ad spend can be calculated. But yes, the question still is: does it raise intent or did the user already want the thing, googled it, and clicked on the first result (which is your ad, above the organic results that also lead to your online store)?
  - dlp211 6y ago
    The fundamental problem with this is that if I bought your product, it's because I was looking for your product after doing a decent amount of research which meant I was on a bunch of retargeting lists and you've wasted money on me. I have never to my recollection purchased anything because of an ad. If I have clicked through an ad, it was simply because it was the quickest way to get to the website/product. There is a role for advertising because Google polluted their organic search results with Ads making it harder for you to find what you want and forcing you to compete for the top Ad spot. I realize that I can only speak for myself, but I have never seen any evidence that advertising works and I've worked in ad tech.
    - ipsum2 6y ago
      It's pretty clear to see direct causation when using ads to promote your product. Promote your product using ads for 1 month, then turn off ads and see how many sales you get. For a majority of e-commerce, you'll see a substantial drop.