9 ms·

>In the end I decided the only real way to tell if what we were doing was working, was to turn it all off, for several months to see if there was a measurable d

>In the end I decided the only real way to tell if what we were doing was working, was to turn it all off, for several months to see if there was a measurable difference. What? If this person genuinely thinks that there is no way to measure an ad's performance while it is running, they are either too ignorant to be in business, or they hired someone who lied to them.

yojo 4y ago
If you’re full e-commerce the only metric that really matters is conversions. How do you know your ads aren’t just taking credit for customers who were going to buy anyway? You can get great CTR and conversion rate running ads against your brand name. But if your SEO is sound you’re pretty much just canibalizing your organic traffic. I’m not aware of a way to determine whether an ad just claimed credit for a customer it didn’t earn. I’m interested to learn though.
- tetsusaiga 4y ago
  I agree that only conversions matter, but you should be able to track whether a customer originated from an ad click vs. SEO, whether they clicked one ad and had 4 sessions since, etc. While it's not a perfect system, you should know with a reasonable degree of accuracy how many dollars you get back for every dollar you put in to ad spend. As far as the SEO traffic issue, this can also be accounted for once you have scaled your ads beyond the levels of what SEO could yield. At low budgets this can be more of an issue, at high budgets much less so.
  - Nextgrid 4y ago
    > whether they clicked one ad and had 4 sessions since, etc. That's not trivial to do, both from a technical point of view (browsers - rightfully - fight these kinds of tracking attempts) but also legal (GDPR mandates that the customer opts into this kind of tracking but they have no incentive to do so).
    - tetsusaiga 4y ago
      You're right, this is actually a lot harder to recently, with the iOS updates that occurred a year or so ago. It's a space that evolves very quickly. But this is a good place to point out that, yes - there is nuance to all of this - and I didn't really mean to turn it into a thesis on online advertising (lol) so much as to say: It's still easy to be more accurate than "Turn it off and see what happens". Even a semi-sophisticated media buyer is going in with a plan, and some method of measurement.
    - m3047 4y ago
      GDPR doesn't stop me from having ad1.example.com and ad2.example.com landing pages. Aspirationally perhaps, but technically no. I think this is different from fonts.gstatic.com in that it doesn't need to follow people around the internet and it's also not info necessarily going to a third party.
  - yojo 4y ago
    I don’t know how counting sessions or source of click matters. You still don’t know whether the customer would have ultimately found you without the ad. Put another way, the only way to tell if a customer will still find you without the ad is to not run the ad. If you have sufficient traffic you might be able to measure this by turning your ad spend down rather than off and measure the bottom line impact. But other than gross spend changes there’s no way of telling for sure that the ads do anything.
    - philote 4y ago
      This is like the paid search results that come up when you search for a specific company name. The company's site is just below those paid results, but often the same site is in the paid results but they are further up so are more likely to be clicked.
    - tetsusaiga 4y ago
      > You still don’t know whether the customer would have ultimately found you without the ad. Sure, maybe 100 customers would eventually find you. But if you buy the traffic, you can make them all find you on the same day. > But other than gross spend changes there’s no way of telling for sure that the ads do anything. I'll use the most fundamental example: If someone clicks your ad and buys the product during that session, you know the ad worked. If you keep increasing your ad budget every day, and you are consistently returning $2 for every $1 you spend, you know it's working. Turn off the ads, and the revenue goes away. You'd be surprised how many people make their living doing this.
      - yobbo 4y ago
        > [...] Turn off the ads, and the revenue goes away. This is the experiment from the article. It showed that ads did not affect revenue.
      - thrwy_ywrht 4y ago
        >If someone clicks your ad and buys the product during that session, you know the ad worked This is not as clear as you make it seem. Let's say there's a hypothetical product that a consumer only buys once every 5 years. If someone clicks your ad and then immediately buys the product... what if they would have bought it anyway, tomorrow, or 5 minutes from now, without the ad? How can you test that counterfactual? If I buy cat litter online once a month every month for 5 years by going to example.org/catlitter -- but then they decide to start advertising on facebook, so now I click the facebook ad once a month to buy cat litter from the same site, are the ads "working"?
  - Strom 4y ago
    > you should be able to track whether a customer originated from an ad click vs. SEO That's easy to track, but answers the wrong question. It answers whether the person clicked on the ad or the organic link. What we actually want to know is whether this person would convert even without seeing an ad. The only way to answer whether a person would still convert without seeing an ad is to make sure that they don't see an ad.
    - tetsusaiga 4y ago
      My major failure in this thread was not specifying from that start that I'm not so much talking about paid search ads (google ads), as much as I mean FB, YouTube, etc. That's the arena where you've really gotta question if you're competing with yourself. On the flip side of that, most of HN seems to think purely in terms of paid search ads, when there are many other types of online advertising that exist, and that don't have this issue baked into them. Such is the nature of internet dialogue, I guess lol.
      - Strom 4y ago
        That's a good point. It's a lot harder to cannibalize organic traffic with non-search ads.
      - m3047 4y ago
        Are you conflating a call to action on a channel you already control with advertising? Because in that case you might try f(A), f(A,B) and also f(B): could be a synergistic effect, or they could just click on anything.
  - Closi 4y ago
    > I agree that only conversions matter, but you should be able to track whether a customer originated from an ad click vs. SEO, whether they clicked one ad and had 4 sessions since, etc. An ad-click could have been an organic click if the ad wasn't there though, which is where the complexity is. The ad might have great conversion, but if the customer would have clicked an organic link that navigated to your site anyway then the ad is taking credit for an organic sale.
    - tetsusaiga 4y ago
      While you're not necessarily wrong, the distinction lies in how many people see you organically. If I get 100 organic impressions/day, and then spend $N to get 500/day, I'm speeding the process up, and nearly guaranteeing to get in front of people who would never see me organically. *Note: I'm largely not talking about paid search ads. Those are definitely an area where you can end up competing with yourself. Sorry to anyone who I replied to earlier, and wasn't clear enough on this with.
      - deleted 4y ago
        [deleted]
- boplicity 4y ago
  > How do you know your ads aren’t just taking credit for customers who were going to buy anyway? If you can't answer that question, then you're not running a proper advertising campaign. You're just throwing money around blindly. Plenty of people do have good answers to that question, though. Unfortunately, plenty of people don't.
  - Strom 4y ago
    > If you can't answer that question, then you're not running a proper advertising campaign. What is this proper advertising campaign? Does it include turning off ads for terms that rank organically high anyway?
    - boplicity 4y ago
      > What is this proper advertising campaign Well, for our business, proper advertising campaigns have opened new channels for growth, allowing us to reach new people, and build our email list significantly, with a predictable ROI. It's pretty easy to tell if an ad is working when you turn it on, and your email list growth rate instantly doubles -- and all the traffic is coming from a new traffic source. (Obviously, if we were to run Google ads on search traffic, we would check whether our organic search traffic dropped when we turned the ads on, but that's a different point.)
      - Strom 4y ago
        > It's pretty easy to tell if an ad is working when you turn it on, and your email list growth rate instantly doubles -- and all the traffic is coming from a new traffic source. Maybe for a real tight definition of instantly, but generally this is not true. Imagine the following scenario: 1. You run an ad for ComapnyName. 2. An enthusiastic customer promotes your business in a local bar. 3. The bar attendees search for CompanyName and click on the first result. Because you ran an ad for CompanyName, the first result is going to be your ad. You will see very nice ROI on that ad. What you won't see in any stats, is that these people would have probably found the organic link anyway, because they were already motivated and searching for CompanyName in particular. > Obviously, if we were to run Google ads on search traffic, we would check whether our organic search traffic dropped when we turned the ads on, but that's a different point That wouldn't help either in this imaginary scenario, because these bar attendees are a spike in traffic. Plus if the ad has been running for a longer period, then you won't have accurate organic search traffic stats anymore either, because it's already cannibalized.
- jefftk 4y ago
  > How do you know your ads aren’t just taking credit for customers who were going to buy anyway? The word for this in the business is "incrementality", and there are several ways of measuring it. The simplest conceptually, is that you run two ads to two random groups of users: one for the product and one for something irrelevant like a charity. Then you compare conversions between the two groups. (There are fancier ways to do it that don't require you to spend half your budget on an irrelevant ad, but that's the basic idea.)
  - whimsicalism 4y ago
    I work in ads. You appear to be one of the few people on this thread who actually knows what they're talking about. +1 to everything you've said.
    - deleted 4y ago
      [deleted]
  - yojo 4y ago
    Thank you. I wasn’t aware of this, it makes perfect sense.
  - yobbo 4y ago
    > is that you show two ads: one for the product and one for something irrelevant like a charity. This is would be another experiment that does not test what you need to in order to demonstrate some sort of causality.
    - jefftk 4y ago
      Why doesn't that demonstrate causality? It is a randomized controlled trial.
      - yobbo 4y ago
        The control would be to not show ads. This was tried in the article, with time as the variable to partition the control groups.
- dangerface 4y ago
  > How do you know your ads aren’t just taking credit for customers who were going to buy anyway? If you are doing advertising right you build a funnel from awareness through to conversion and track every part of it so you then a/b test your adverts and channels. Getting advert channel fit is the key to success and can only be done by testing and measuring your adverts. The fact that op couldn't measure the effectiveness of their adverts shows they weren't doing this and their bad outcome should be 100% expected. If you are just buying adverts in the hope you get more conversions then you are putting those adverts out to die. Burn your money you will probably get more eyes for doing that than untested adverts.
  - whimsicalism 4y ago
    This is now impossible to do on iOS devices.
    - dangerface 4y ago
      How is it impossible on iOS?
      - whimsicalism 4y ago
        Click attribution is basically not possible due to new Apple policy on tracking, unless you opt-in.
bschne 4y ago
Well that works in some ways, but it can also be very hard to disentangle the actual causal impact of an ad campaign! Absent running an experiment, you don't really know if people would or would not have bought through some other channel without your campaign. The most striking example is probably the experiment Tadelis convinced eBay to run on their brand-name ads, though the subtlety applies to less obvious cases of questionable ad spend too — https://faculty.haas.berkeley.edu/stadelis/Tadelis.pdf https://faculty.haas.berkeley.edu/stadelis/Tadelis.pdf
- tetsusaiga 4y ago
  I'll just reference my other replies to the first part here, but thank you for that link too. I'll have to give it a read when I have more time later today
civilized 4y ago
One problem is the ad industry is quite good at finding ways to take credit for conversions that would have happened regardless, and analytics are all systematically biased in a way that conveniently maximizes the ability to do so. The idea to simply turn off ads and see the effect is born of a healthy distrust of ad analytics industry bluster.
- josefresco 4y ago
  > One problem is the ad industry is quite good at finding ways to take credit for conversions that would have happened regardless This is a great counter-point when someone mentions "use conversion tracking!" Conversion tracking is great, but not if they store a cookie for 7/15/30 days and "award" the conversion to the ad, when the customer took a different and varying path to purchase. Sure the ad contributed "some" to the conversion, but not 100%.
  - civilized 4y ago
    I strongly suspect they sometimes show the ad and credit it for the conversion after the user shows initial interest in some specific product, i.e. when the user was already highly likely to buy before seeing the ad. This would explain why you're often flooded with ads for something after you search for it, go to its website, or purchase something there. (We've all had that experience of buying something and then being tailed by ads for that thing for weeks afterwards, even though no sane person would think we were going to buy that thing again in such a short time frame.) A story to illustrate: there was once a pizza store that had two guys go out into the city to distribute promotional coupons. The coupons had codes on them so the business could attribute sales to each coupon distributor. John went out into the city and tried his best to drum up new business. Chad stood next to the door of the pizza place and handed a coupon to anyone who was walking in. After a month, 98% of the coupons used were from Chad. Chad got a big bonus and John was let go.
deleted 4y ago
[deleted]
yobbo 4y ago
> this person genuinely thinks that there is no way to measure an ad's performance while This is the case in many situations. The time between initial visit to site to conversion can be weeks, months, or years. Meanwhile, customers will visit site from various platforms. There is no way of attributing value to a single click, it needs to be approximated as an aggregate.
LightHugger 4y ago
Companies that sell ads will do everything they can to convince you there is a measurable difference, but often while the metrics presented to the client are real, they do not always translate to real world impact. There is no easy way to determine whether this is the case without just comparing when the ads are on and off and somehow dealing with the confounding variables, which isn't easy.
gk1 4y ago
Don't pretend attribution is simple or solved. If that were true there wouldn't be dozens of attribution products, who are still far from perfect. The Dunning Kruger effect is strong in marketing. The more you learn about marketing the more reason you have to be unsure.
m3047 4y ago
_How would you measure this:_ Does this ad improve conversion compared to not having the ad at all?
mbesto 4y ago
> If this person genuinely thinks that there is no way to measure an ad's performance while it is running, they are either too ignorant to be in business, or they hired someone who lied to them. Doing a blackout month is a legitimate technique. I've deeply studied ad performance metrics and they are NOT as conclusive as people might think. If they were, you could just keep increasing the amount of $ spent and the conversions would go up. Ok, not exactly, there is usually a diminishing returns aspect, but you get the point. A great example of why this requires more analysis is branded vs non-branded search terms on AdWords. Let's say you spend $10k/month on the brand "Mattel" (aka branded) and $10k/month on "toys" (aka non-branded). Mattel likely gets you like a 20x ROAS because people are probably searching for a specific Mattel toy. It's also potentially likely that if you completely turned off branded search you would net the same results. Ad attribution is NOTORIOUSLY difficult: https://www.optimizesmart.com/what-is-attribution-problem-in-online-marketing/ https://www.optimizesmart.com/what-is-attribution-problem-in... so to pretend like "this guy is an idiot, he doesn't know what he's doing" is disingenuous.

yojo 4y ago

If you’re full e-commerce the only metric that really matters is conversions. How do you know your ads aren’t just taking credit for customers who were going to buy anyway? You can get great CTR and conversion rate running ads against your brand name. But if your SEO is sound you’re pretty much just canibalizing your organic traffic. I’m not aware of a way to determine whether an ad just claimed credit for a customer it didn’t earn. I’m interested to learn though.

tetsusaiga 4y ago
I agree that only conversions matter, but you should be able to track whether a customer originated from an ad click vs. SEO, whether they clicked one ad and had 4 sessions since, etc. While it's not a perfect system, you should know with a reasonable degree of accuracy how many dollars you get back for every dollar you put in to ad spend. As far as the SEO traffic issue, this can also be accounted for once you have scaled your ads beyond the levels of what SEO could yield. At low budgets this can be more of an issue, at high budgets much less so.
- Nextgrid 4y ago
  > whether they clicked one ad and had 4 sessions since, etc. That's not trivial to do, both from a technical point of view (browsers - rightfully - fight these kinds of tracking attempts) but also legal (GDPR mandates that the customer opts into this kind of tracking but they have no incentive to do so).
  - tetsusaiga 4y ago
    You're right, this is actually a lot harder to recently, with the iOS updates that occurred a year or so ago. It's a space that evolves very quickly. But this is a good place to point out that, yes - there is nuance to all of this - and I didn't really mean to turn it into a thesis on online advertising (lol) so much as to say: It's still easy to be more accurate than "Turn it off and see what happens". Even a semi-sophisticated media buyer is going in with a plan, and some method of measurement.
  - m3047 4y ago
    GDPR doesn't stop me from having ad1.example.com and ad2.example.com landing pages. Aspirationally perhaps, but technically no. I think this is different from fonts.gstatic.com in that it doesn't need to follow people around the internet and it's also not info necessarily going to a third party.
- yojo 4y ago
  I don’t know how counting sessions or source of click matters. You still don’t know whether the customer would have ultimately found you without the ad. Put another way, the only way to tell if a customer will still find you without the ad is to not run the ad. If you have sufficient traffic you might be able to measure this by turning your ad spend down rather than off and measure the bottom line impact. But other than gross spend changes there’s no way of telling for sure that the ads do anything.
  - philote 4y ago
    This is like the paid search results that come up when you search for a specific company name. The company's site is just below those paid results, but often the same site is in the paid results but they are further up so are more likely to be clicked.
  - tetsusaiga 4y ago
    > You still don’t know whether the customer would have ultimately found you without the ad. Sure, maybe 100 customers would eventually find you. But if you buy the traffic, you can make them all find you on the same day. > But other than gross spend changes there’s no way of telling for sure that the ads do anything. I'll use the most fundamental example: If someone clicks your ad and buys the product during that session, you know the ad worked. If you keep increasing your ad budget every day, and you are consistently returning $2 for every $1 you spend, you know it's working. Turn off the ads, and the revenue goes away. You'd be surprised how many people make their living doing this.
    - yobbo 4y ago
      > [...] Turn off the ads, and the revenue goes away. This is the experiment from the article. It showed that ads did not affect revenue.
    - thrwy_ywrht 4y ago
      >If someone clicks your ad and buys the product during that session, you know the ad worked This is not as clear as you make it seem. Let's say there's a hypothetical product that a consumer only buys once every 5 years. If someone clicks your ad and then immediately buys the product... what if they would have bought it anyway, tomorrow, or 5 minutes from now, without the ad? How can you test that counterfactual? If I buy cat litter online once a month every month for 5 years by going to example.org/catlitter -- but then they decide to start advertising on facebook, so now I click the facebook ad once a month to buy cat litter from the same site, are the ads "working"?
- Strom 4y ago
  > you should be able to track whether a customer originated from an ad click vs. SEO That's easy to track, but answers the wrong question. It answers whether the person clicked on the ad or the organic link. What we actually want to know is whether this person would convert even without seeing an ad. The only way to answer whether a person would still convert without seeing an ad is to make sure that they don't see an ad.
  - tetsusaiga 4y ago
    My major failure in this thread was not specifying from that start that I'm not so much talking about paid search ads (google ads), as much as I mean FB, YouTube, etc. That's the arena where you've really gotta question if you're competing with yourself. On the flip side of that, most of HN seems to think purely in terms of paid search ads, when there are many other types of online advertising that exist, and that don't have this issue baked into them. Such is the nature of internet dialogue, I guess lol.
    - Strom 4y ago
      That's a good point. It's a lot harder to cannibalize organic traffic with non-search ads.
    - m3047 4y ago
      Are you conflating a call to action on a channel you already control with advertising? Because in that case you might try f(A), f(A,B) and also f(B): could be a synergistic effect, or they could just click on anything.
- Closi 4y ago
  > I agree that only conversions matter, but you should be able to track whether a customer originated from an ad click vs. SEO, whether they clicked one ad and had 4 sessions since, etc. An ad-click could have been an organic click if the ad wasn't there though, which is where the complexity is. The ad might have great conversion, but if the customer would have clicked an organic link that navigated to your site anyway then the ad is taking credit for an organic sale.
  - tetsusaiga 4y ago
    While you're not necessarily wrong, the distinction lies in how many people see you organically. If I get 100 organic impressions/day, and then spend $N to get 500/day, I'm speeding the process up, and nearly guaranteeing to get in front of people who would never see me organically. *Note: I'm largely not talking about paid search ads. Those are definitely an area where you can end up competing with yourself. Sorry to anyone who I replied to earlier, and wasn't clear enough on this with.
    - deleted 4y ago
      [deleted]
boplicity 4y ago
> How do you know your ads aren’t just taking credit for customers who were going to buy anyway? If you can't answer that question, then you're not running a proper advertising campaign. You're just throwing money around blindly. Plenty of people do have good answers to that question, though. Unfortunately, plenty of people don't.
- Strom 4y ago
  > If you can't answer that question, then you're not running a proper advertising campaign. What is this proper advertising campaign? Does it include turning off ads for terms that rank organically high anyway?
  - boplicity 4y ago
    > What is this proper advertising campaign Well, for our business, proper advertising campaigns have opened new channels for growth, allowing us to reach new people, and build our email list significantly, with a predictable ROI. It's pretty easy to tell if an ad is working when you turn it on, and your email list growth rate instantly doubles -- and all the traffic is coming from a new traffic source. (Obviously, if we were to run Google ads on search traffic, we would check whether our organic search traffic dropped when we turned the ads on, but that's a different point.)
    - Strom 4y ago
      > It's pretty easy to tell if an ad is working when you turn it on, and your email list growth rate instantly doubles -- and all the traffic is coming from a new traffic source. Maybe for a real tight definition of instantly, but generally this is not true. Imagine the following scenario: 1. You run an ad for ComapnyName. 2. An enthusiastic customer promotes your business in a local bar. 3. The bar attendees search for CompanyName and click on the first result. Because you ran an ad for CompanyName, the first result is going to be your ad. You will see very nice ROI on that ad. What you won't see in any stats, is that these people would have probably found the organic link anyway, because they were already motivated and searching for CompanyName in particular. > Obviously, if we were to run Google ads on search traffic, we would check whether our organic search traffic dropped when we turned the ads on, but that's a different point That wouldn't help either in this imaginary scenario, because these bar attendees are a spike in traffic. Plus if the ad has been running for a longer period, then you won't have accurate organic search traffic stats anymore either, because it's already cannibalized.
jefftk 4y ago
> How do you know your ads aren’t just taking credit for customers who were going to buy anyway? The word for this in the business is "incrementality", and there are several ways of measuring it. The simplest conceptually, is that you run two ads to two random groups of users: one for the product and one for something irrelevant like a charity. Then you compare conversions between the two groups. (There are fancier ways to do it that don't require you to spend half your budget on an irrelevant ad, but that's the basic idea.)
- whimsicalism 4y ago
  I work in ads. You appear to be one of the few people on this thread who actually knows what they're talking about. +1 to everything you've said.
  - deleted 4y ago
    [deleted]
- yojo 4y ago
  Thank you. I wasn’t aware of this, it makes perfect sense.
- yobbo 4y ago
  > is that you show two ads: one for the product and one for something irrelevant like a charity. This is would be another experiment that does not test what you need to in order to demonstrate some sort of causality.
  - jefftk 4y ago
    Why doesn't that demonstrate causality? It is a randomized controlled trial.
    - yobbo 4y ago
      The control would be to not show ads. This was tried in the article, with time as the variable to partition the control groups.
dangerface 4y ago
> How do you know your ads aren’t just taking credit for customers who were going to buy anyway? If you are doing advertising right you build a funnel from awareness through to conversion and track every part of it so you then a/b test your adverts and channels. Getting advert channel fit is the key to success and can only be done by testing and measuring your adverts. The fact that op couldn't measure the effectiveness of their adverts shows they weren't doing this and their bad outcome should be 100% expected. If you are just buying adverts in the hope you get more conversions then you are putting those adverts out to die. Burn your money you will probably get more eyes for doing that than untested adverts.
- whimsicalism 4y ago
  This is now impossible to do on iOS devices.
  - dangerface 4y ago
    How is it impossible on iOS?
    - whimsicalism 4y ago
      Click attribution is basically not possible due to new Apple policy on tracking, unless you opt-in.

bschne 4y ago

Well that works in some ways, but it can also be very hard to disentangle the actual causal impact of an ad campaign! Absent running an experiment, you don't really know if people would or would not have bought through some other channel without your campaign. The most striking example is probably the experiment Tadelis convinced eBay to run on their brand-name ads, though the subtlety applies to less obvious cases of questionable ad spend too — https://faculty.haas.berkeley.edu/stadelis/Tadelis.pdf https://faculty.haas.berkeley.edu/stadelis/Tadelis.pdf

tetsusaiga 4y ago
I'll just reference my other replies to the first part here, but thank you for that link too. I'll have to give it a read when I have more time later today

civilized 4y ago

One problem is the ad industry is quite good at finding ways to take credit for conversions that would have happened regardless, and analytics are all systematically biased in a way that conveniently maximizes the ability to do so. The idea to simply turn off ads and see the effect is born of a healthy distrust of ad analytics industry bluster.

josefresco 4y ago
> One problem is the ad industry is quite good at finding ways to take credit for conversions that would have happened regardless This is a great counter-point when someone mentions "use conversion tracking!" Conversion tracking is great, but not if they store a cookie for 7/15/30 days and "award" the conversion to the ad, when the customer took a different and varying path to purchase. Sure the ad contributed "some" to the conversion, but not 100%.
- civilized 4y ago
  I strongly suspect they sometimes show the ad and credit it for the conversion after the user shows initial interest in some specific product, i.e. when the user was already highly likely to buy before seeing the ad. This would explain why you're often flooded with ads for something after you search for it, go to its website, or purchase something there. (We've all had that experience of buying something and then being tailed by ads for that thing for weeks afterwards, even though no sane person would think we were going to buy that thing again in such a short time frame.) A story to illustrate: there was once a pizza store that had two guys go out into the city to distribute promotional coupons. The coupons had codes on them so the business could attribute sales to each coupon distributor. John went out into the city and tried his best to drum up new business. Chad stood next to the door of the pizza place and handed a coupon to anyone who was walking in. After a month, 98% of the coupons used were from Chad. Chad got a big bonus and John was let go.

deleted 4y ago

[deleted]

yobbo 4y ago

> this person genuinely thinks that there is no way to measure an ad's performance while This is the case in many situations. The time between initial visit to site to conversion can be weeks, months, or years. Meanwhile, customers will visit site from various platforms. There is no way of attributing value to a single click, it needs to be approximated as an aggregate.

LightHugger 4y ago

Companies that sell ads will do everything they can to convince you there is a measurable difference, but often while the metrics presented to the client are real, they do not always translate to real world impact. There is no easy way to determine whether this is the case without just comparing when the ads are on and off and somehow dealing with the confounding variables, which isn't easy.

gk1 4y ago

Don't pretend attribution is simple or solved. If that were true there wouldn't be dozens of attribution products, who are still far from perfect. The Dunning Kruger effect is strong in marketing. The more you learn about marketing the more reason you have to be unsure.

m3047 4y ago

_How would you measure this:_ Does this ad improve conversion compared to not having the ad at all?

mbesto 4y ago

> If this person genuinely thinks that there is no way to measure an ad's performance while it is running, they are either too ignorant to be in business, or they hired someone who lied to them. Doing a blackout month is a legitimate technique. I've deeply studied ad performance metrics and they are NOT as conclusive as people might think. If they were, you could just keep increasing the amount of $ spent and the conversions would go up. Ok, not exactly, there is usually a diminishing returns aspect, but you get the point. A great example of why this requires more analysis is branded vs non-branded search terms on AdWords. Let's say you spend $10k/month on the brand "Mattel" (aka branded) and $10k/month on "toys" (aka non-branded). Mattel likely gets you like a 20x ROAS because people are probably searching for a specific Mattel toy. It's also potentially likely that if you completely turned off branded search you would net the same results. Ad attribution is NOTORIOUSLY difficult: https://www.optimizesmart.com/what-is-attribution-problem-in-online-marketing/ https://www.optimizesmart.com/what-is-attribution-problem-in... so to pretend like "this guy is an idiot, he doesn't know what he's doing" is disingenuous.