9 ms·

If you’re full e-commerce the only metric that really matters is conversions. How do you know your ads aren’t just taking credit for customers who were going to

by yojo 5y ago

If you’re full e-commerce the only metric that really matters is conversions. How do you know your ads aren’t just taking credit for customers who were going to buy anyway? You can get great CTR and conversion rate running ads against your brand name. But if your SEO is sound you’re pretty much just canibalizing your organic traffic. I’m not aware of a way to determine whether an ad just claimed credit for a customer it didn’t earn. I’m interested to learn though.

tetsusaiga 5y ago
I agree that only conversions matter, but you should be able to track whether a customer originated from an ad click vs. SEO, whether they clicked one ad and had 4 sessions since, etc. While it's not a perfect system, you should know with a reasonable degree of accuracy how many dollars you get back for every dollar you put in to ad spend. As far as the SEO traffic issue, this can also be accounted for once you have scaled your ads beyond the levels of what SEO could yield. At low budgets this can be more of an issue, at high budgets much less so.
- Nextgrid 5y ago
  > whether they clicked one ad and had 4 sessions since, etc. That's not trivial to do, both from a technical point of view (browsers - rightfully - fight these kinds of tracking attempts) but also legal (GDPR mandates that the customer opts into this kind of tracking but they have no incentive to do so).
  - tetsusaiga 5y ago
    You're right, this is actually a lot harder to recently, with the iOS updates that occurred a year or so ago. It's a space that evolves very quickly. But this is a good place to point out that, yes - there is nuance to all of this - and I didn't really mean to turn it into a thesis on online advertising (lol) so much as to say: It's still easy to be more accurate than "Turn it off and see what happens". Even a semi-sophisticated media buyer is going in with a plan, and some method of measurement.
  - m3047 5y ago
    GDPR doesn't stop me from having ad1.example.com and ad2.example.com landing pages. Aspirationally perhaps, but technically no. I think this is different from fonts.gstatic.com in that it doesn't need to follow people around the internet and it's also not info necessarily going to a third party.
- yojo 5y ago
  I don’t know how counting sessions or source of click matters. You still don’t know whether the customer would have ultimately found you without the ad. Put another way, the only way to tell if a customer will still find you without the ad is to not run the ad. If you have sufficient traffic you might be able to measure this by turning your ad spend down rather than off and measure the bottom line impact. But other than gross spend changes there’s no way of telling for sure that the ads do anything.
  - philote 5y ago
    This is like the paid search results that come up when you search for a specific company name. The company's site is just below those paid results, but often the same site is in the paid results but they are further up so are more likely to be clicked.
  - tetsusaiga 5y ago
    > You still don’t know whether the customer would have ultimately found you without the ad. Sure, maybe 100 customers would eventually find you. But if you buy the traffic, you can make them all find you on the same day. > But other than gross spend changes there’s no way of telling for sure that the ads do anything. I'll use the most fundamental example: If someone clicks your ad and buys the product during that session, you know the ad worked. If you keep increasing your ad budget every day, and you are consistently returning $2 for every $1 you spend, you know it's working. Turn off the ads, and the revenue goes away. You'd be surprised how many people make their living doing this.
    - yobbo 5y ago
      > [...] Turn off the ads, and the revenue goes away. This is the experiment from the article. It showed that ads did not affect revenue.
    - thrwy_ywrht 5y ago
      >If someone clicks your ad and buys the product during that session, you know the ad worked This is not as clear as you make it seem. Let's say there's a hypothetical product that a consumer only buys once every 5 years. If someone clicks your ad and then immediately buys the product... what if they would have bought it anyway, tomorrow, or 5 minutes from now, without the ad? How can you test that counterfactual? If I buy cat litter online once a month every month for 5 years by going to example.org/catlitter -- but then they decide to start advertising on facebook, so now I click the facebook ad once a month to buy cat litter from the same site, are the ads "working"?
- Strom 5y ago
  > you should be able to track whether a customer originated from an ad click vs. SEO That's easy to track, but answers the wrong question. It answers whether the person clicked on the ad or the organic link. What we actually want to know is whether this person would convert even without seeing an ad. The only way to answer whether a person would still convert without seeing an ad is to make sure that they don't see an ad.
  - tetsusaiga 5y ago
    My major failure in this thread was not specifying from that start that I'm not so much talking about paid search ads (google ads), as much as I mean FB, YouTube, etc. That's the arena where you've really gotta question if you're competing with yourself. On the flip side of that, most of HN seems to think purely in terms of paid search ads, when there are many other types of online advertising that exist, and that don't have this issue baked into them. Such is the nature of internet dialogue, I guess lol.
    - Strom 5y ago
      That's a good point. It's a lot harder to cannibalize organic traffic with non-search ads.
    - m3047 5y ago
      Are you conflating a call to action on a channel you already control with advertising? Because in that case you might try f(A), f(A,B) and also f(B): could be a synergistic effect, or they could just click on anything.
- Closi 5y ago
  > I agree that only conversions matter, but you should be able to track whether a customer originated from an ad click vs. SEO, whether they clicked one ad and had 4 sessions since, etc. An ad-click could have been an organic click if the ad wasn't there though, which is where the complexity is. The ad might have great conversion, but if the customer would have clicked an organic link that navigated to your site anyway then the ad is taking credit for an organic sale.
  - tetsusaiga 5y ago
    While you're not necessarily wrong, the distinction lies in how many people see you organically. If I get 100 organic impressions/day, and then spend $N to get 500/day, I'm speeding the process up, and nearly guaranteeing to get in front of people who would never see me organically. *Note: I'm largely not talking about paid search ads. Those are definitely an area where you can end up competing with yourself. Sorry to anyone who I replied to earlier, and wasn't clear enough on this with.
    - deleted 5y ago
      [deleted]
boplicity 5y ago
> How do you know your ads aren’t just taking credit for customers who were going to buy anyway? If you can't answer that question, then you're not running a proper advertising campaign. You're just throwing money around blindly. Plenty of people do have good answers to that question, though. Unfortunately, plenty of people don't.
- Strom 5y ago
  > If you can't answer that question, then you're not running a proper advertising campaign. What is this proper advertising campaign? Does it include turning off ads for terms that rank organically high anyway?
  - boplicity 5y ago
    > What is this proper advertising campaign Well, for our business, proper advertising campaigns have opened new channels for growth, allowing us to reach new people, and build our email list significantly, with a predictable ROI. It's pretty easy to tell if an ad is working when you turn it on, and your email list growth rate instantly doubles -- and all the traffic is coming from a new traffic source. (Obviously, if we were to run Google ads on search traffic, we would check whether our organic search traffic dropped when we turned the ads on, but that's a different point.)
    - Strom 5y ago
      > It's pretty easy to tell if an ad is working when you turn it on, and your email list growth rate instantly doubles -- and all the traffic is coming from a new traffic source. Maybe for a real tight definition of instantly, but generally this is not true. Imagine the following scenario: 1. You run an ad for ComapnyName. 2. An enthusiastic customer promotes your business in a local bar. 3. The bar attendees search for CompanyName and click on the first result. Because you ran an ad for CompanyName, the first result is going to be your ad. You will see very nice ROI on that ad. What you won't see in any stats, is that these people would have probably found the organic link anyway, because they were already motivated and searching for CompanyName in particular. > Obviously, if we were to run Google ads on search traffic, we would check whether our organic search traffic dropped when we turned the ads on, but that's a different point That wouldn't help either in this imaginary scenario, because these bar attendees are a spike in traffic. Plus if the ad has been running for a longer period, then you won't have accurate organic search traffic stats anymore either, because it's already cannibalized.
jefftk 5y ago
> How do you know your ads aren’t just taking credit for customers who were going to buy anyway? The word for this in the business is "incrementality", and there are several ways of measuring it. The simplest conceptually, is that you run two ads to two random groups of users: one for the product and one for something irrelevant like a charity. Then you compare conversions between the two groups. (There are fancier ways to do it that don't require you to spend half your budget on an irrelevant ad, but that's the basic idea.)
- whimsicalism 5y ago
  I work in ads. You appear to be one of the few people on this thread who actually knows what they're talking about. +1 to everything you've said.
  - deleted 5y ago
    [deleted]
- yojo 5y ago
  Thank you. I wasn’t aware of this, it makes perfect sense.
- yobbo 5y ago
  > is that you show two ads: one for the product and one for something irrelevant like a charity. This is would be another experiment that does not test what you need to in order to demonstrate some sort of causality.
  - jefftk 5y ago
    Why doesn't that demonstrate causality? It is a randomized controlled trial.
    - yobbo 5y ago
      The control would be to not show ads. This was tried in the article, with time as the variable to partition the control groups.
dangerface 5y ago
> How do you know your ads aren’t just taking credit for customers who were going to buy anyway? If you are doing advertising right you build a funnel from awareness through to conversion and track every part of it so you then a/b test your adverts and channels. Getting advert channel fit is the key to success and can only be done by testing and measuring your adverts. The fact that op couldn't measure the effectiveness of their adverts shows they weren't doing this and their bad outcome should be 100% expected. If you are just buying adverts in the hope you get more conversions then you are putting those adverts out to die. Burn your money you will probably get more eyes for doing that than untested adverts.
- whimsicalism 5y ago
  This is now impossible to do on iOS devices.
  - dangerface 5y ago
    How is it impossible on iOS?
    - whimsicalism 5y ago
      Click attribution is basically not possible due to new Apple policy on tracking, unless you opt-in.

by yojo 5y ago

tetsusaiga 5y ago

I agree that only conversions matter, but you should be able to track whether a customer originated from an ad click vs. SEO, whether they clicked one ad and had 4 sessions since, etc. While it's not a perfect system, you should know with a reasonable degree of accuracy how many dollars you get back for every dollar you put in to ad spend. As far as the SEO traffic issue, this can also be accounted for once you have scaled your ads beyond the levels of what SEO could yield. At low budgets this can be more of an issue, at high budgets much less so.

Nextgrid 5y ago
> whether they clicked one ad and had 4 sessions since, etc. That's not trivial to do, both from a technical point of view (browsers - rightfully - fight these kinds of tracking attempts) but also legal (GDPR mandates that the customer opts into this kind of tracking but they have no incentive to do so).
- tetsusaiga 5y ago
  You're right, this is actually a lot harder to recently, with the iOS updates that occurred a year or so ago. It's a space that evolves very quickly. But this is a good place to point out that, yes - there is nuance to all of this - and I didn't really mean to turn it into a thesis on online advertising (lol) so much as to say: It's still easy to be more accurate than "Turn it off and see what happens". Even a semi-sophisticated media buyer is going in with a plan, and some method of measurement.
- m3047 5y ago
  GDPR doesn't stop me from having ad1.example.com and ad2.example.com landing pages. Aspirationally perhaps, but technically no. I think this is different from fonts.gstatic.com in that it doesn't need to follow people around the internet and it's also not info necessarily going to a third party.
yojo 5y ago
I don’t know how counting sessions or source of click matters. You still don’t know whether the customer would have ultimately found you without the ad. Put another way, the only way to tell if a customer will still find you without the ad is to not run the ad. If you have sufficient traffic you might be able to measure this by turning your ad spend down rather than off and measure the bottom line impact. But other than gross spend changes there’s no way of telling for sure that the ads do anything.
- philote 5y ago
  This is like the paid search results that come up when you search for a specific company name. The company's site is just below those paid results, but often the same site is in the paid results but they are further up so are more likely to be clicked.
- tetsusaiga 5y ago
  > You still don’t know whether the customer would have ultimately found you without the ad. Sure, maybe 100 customers would eventually find you. But if you buy the traffic, you can make them all find you on the same day. > But other than gross spend changes there’s no way of telling for sure that the ads do anything. I'll use the most fundamental example: If someone clicks your ad and buys the product during that session, you know the ad worked. If you keep increasing your ad budget every day, and you are consistently returning $2 for every $1 you spend, you know it's working. Turn off the ads, and the revenue goes away. You'd be surprised how many people make their living doing this.
  - yobbo 5y ago
    > [...] Turn off the ads, and the revenue goes away. This is the experiment from the article. It showed that ads did not affect revenue.
  - thrwy_ywrht 5y ago
    >If someone clicks your ad and buys the product during that session, you know the ad worked This is not as clear as you make it seem. Let's say there's a hypothetical product that a consumer only buys once every 5 years. If someone clicks your ad and then immediately buys the product... what if they would have bought it anyway, tomorrow, or 5 minutes from now, without the ad? How can you test that counterfactual? If I buy cat litter online once a month every month for 5 years by going to example.org/catlitter -- but then they decide to start advertising on facebook, so now I click the facebook ad once a month to buy cat litter from the same site, are the ads "working"?
Strom 5y ago
> you should be able to track whether a customer originated from an ad click vs. SEO That's easy to track, but answers the wrong question. It answers whether the person clicked on the ad or the organic link. What we actually want to know is whether this person would convert even without seeing an ad. The only way to answer whether a person would still convert without seeing an ad is to make sure that they don't see an ad.
- tetsusaiga 5y ago
  My major failure in this thread was not specifying from that start that I'm not so much talking about paid search ads (google ads), as much as I mean FB, YouTube, etc. That's the arena where you've really gotta question if you're competing with yourself. On the flip side of that, most of HN seems to think purely in terms of paid search ads, when there are many other types of online advertising that exist, and that don't have this issue baked into them. Such is the nature of internet dialogue, I guess lol.
  - Strom 5y ago
    That's a good point. It's a lot harder to cannibalize organic traffic with non-search ads.
  - m3047 5y ago
    Are you conflating a call to action on a channel you already control with advertising? Because in that case you might try f(A), f(A,B) and also f(B): could be a synergistic effect, or they could just click on anything.
Closi 5y ago
> I agree that only conversions matter, but you should be able to track whether a customer originated from an ad click vs. SEO, whether they clicked one ad and had 4 sessions since, etc. An ad-click could have been an organic click if the ad wasn't there though, which is where the complexity is. The ad might have great conversion, but if the customer would have clicked an organic link that navigated to your site anyway then the ad is taking credit for an organic sale.
- tetsusaiga 5y ago
  While you're not necessarily wrong, the distinction lies in how many people see you organically. If I get 100 organic impressions/day, and then spend $N to get 500/day, I'm speeding the process up, and nearly guaranteeing to get in front of people who would never see me organically. *Note: I'm largely not talking about paid search ads. Those are definitely an area where you can end up competing with yourself. Sorry to anyone who I replied to earlier, and wasn't clear enough on this with.
  - deleted 5y ago
    [deleted]

boplicity 5y ago

> How do you know your ads aren’t just taking credit for customers who were going to buy anyway? If you can't answer that question, then you're not running a proper advertising campaign. You're just throwing money around blindly. Plenty of people do have good answers to that question, though. Unfortunately, plenty of people don't.

Strom 5y ago
> If you can't answer that question, then you're not running a proper advertising campaign. What is this proper advertising campaign? Does it include turning off ads for terms that rank organically high anyway?
- boplicity 5y ago
  > What is this proper advertising campaign Well, for our business, proper advertising campaigns have opened new channels for growth, allowing us to reach new people, and build our email list significantly, with a predictable ROI. It's pretty easy to tell if an ad is working when you turn it on, and your email list growth rate instantly doubles -- and all the traffic is coming from a new traffic source. (Obviously, if we were to run Google ads on search traffic, we would check whether our organic search traffic dropped when we turned the ads on, but that's a different point.)
  - Strom 5y ago
    > It's pretty easy to tell if an ad is working when you turn it on, and your email list growth rate instantly doubles -- and all the traffic is coming from a new traffic source. Maybe for a real tight definition of instantly, but generally this is not true. Imagine the following scenario: 1. You run an ad for ComapnyName. 2. An enthusiastic customer promotes your business in a local bar. 3. The bar attendees search for CompanyName and click on the first result. Because you ran an ad for CompanyName, the first result is going to be your ad. You will see very nice ROI on that ad. What you won't see in any stats, is that these people would have probably found the organic link anyway, because they were already motivated and searching for CompanyName in particular. > Obviously, if we were to run Google ads on search traffic, we would check whether our organic search traffic dropped when we turned the ads on, but that's a different point That wouldn't help either in this imaginary scenario, because these bar attendees are a spike in traffic. Plus if the ad has been running for a longer period, then you won't have accurate organic search traffic stats anymore either, because it's already cannibalized.

jefftk 5y ago

> How do you know your ads aren’t just taking credit for customers who were going to buy anyway? The word for this in the business is "incrementality", and there are several ways of measuring it. The simplest conceptually, is that you run two ads to two random groups of users: one for the product and one for something irrelevant like a charity. Then you compare conversions between the two groups. (There are fancier ways to do it that don't require you to spend half your budget on an irrelevant ad, but that's the basic idea.)

whimsicalism 5y ago
I work in ads. You appear to be one of the few people on this thread who actually knows what they're talking about. +1 to everything you've said.
- deleted 5y ago
  [deleted]
yojo 5y ago
Thank you. I wasn’t aware of this, it makes perfect sense.
yobbo 5y ago
> is that you show two ads: one for the product and one for something irrelevant like a charity. This is would be another experiment that does not test what you need to in order to demonstrate some sort of causality.
- jefftk 5y ago
  Why doesn't that demonstrate causality? It is a randomized controlled trial.
  - yobbo 5y ago
    The control would be to not show ads. This was tried in the article, with time as the variable to partition the control groups.

dangerface 5y ago

> How do you know your ads aren’t just taking credit for customers who were going to buy anyway? If you are doing advertising right you build a funnel from awareness through to conversion and track every part of it so you then a/b test your adverts and channels. Getting advert channel fit is the key to success and can only be done by testing and measuring your adverts. The fact that op couldn't measure the effectiveness of their adverts shows they weren't doing this and their bad outcome should be 100% expected. If you are just buying adverts in the hope you get more conversions then you are putting those adverts out to die. Burn your money you will probably get more eyes for doing that than untested adverts.

whimsicalism 5y ago
This is now impossible to do on iOS devices.
- dangerface 5y ago
  How is it impossible on iOS?
  - whimsicalism 5y ago
    Click attribution is basically not possible due to new Apple policy on tracking, unless you opt-in.