4 ms·

Not all ads are banner ads, and TV spots are no guarantee you'll have people's attention (not to mention you know less about who those people are). There reaso

Not all ads are banner ads, and TV spots are no guarantee you'll have people's attention (not to mention you know less about who those people are). There reason advertising is hard to measure despite lots of data is that the data isn't always accurate and doesn't always tell the whole picture. When you do advertising for a big brand, things are even harder to measure than direct response advertising. Viewability is a problem, but attribution is an even bigger one because your tracking to the end purchase is missing many key data points. Hope that helps clarify a bit.

mkagenius 9y ago
I do not understand why it’s hard to see at least the end result. Before switching on the ads and after switching on the ads. I am sure a company as big as proctor and gamble has at least a team of people who are dedicated to gather the end to end metric. Then where’s the gap?
- ACow_Adonis 9y ago
  One big part of the problem is whether that team (if they exist at all, and that's a big IF) works for the marketing department and whether they have an inherent conflict of interest. In my experience as a data scientist working along-side some marketing people, most of the performance measuring, if it was ever done, was total psuedo-science. It was almost universally done either by people aligned with the actual marketing or who actually worked on said marketing effort. Not only is there the universal mantra of "Don't ever come to a finding which says that your team and effort fails to contribute value to the company", but lets say that you actually are a data scientist and you really are independent and you come to a conclusion that your multi-million dollar marketing campaign is doing sweet f all. This will generally be despite 6 months of presentations from the marketing team about how great it was, and probably several bonuses/pay reviews. And now you're not only fighting a civil war in your company, you're doing it against multiple people with millions of dollars in budget, who have been proclaiming their success, and who's entire career has been based upon appealing to peoples guts to give them high paying jobs and big budgets. Good luck with that fight... Rule 37 of professional analytics in big companies: be VERY careful before turning your analytical eye inwards into your own companies' hierarchy/practices...
  - wastedhours 9y ago
    I think we're reaching that crux point now - it's always been a black box of advertising online, but now teams are taking some knowledge in-house, there's increasing accountability, and thus cutting of budgets as they're clocking onto ROI (which a lot of channels can't show).
- matt4077 9y ago
  Your decision to buy Nike sneakers might be influenced by Nike's sponsorship of Uruguay in the 1998 World Cup. Maybe you were a child back then, and thought their goalkeeper was just the coolest dude, ever. See where the problem might be?
  - nekopa 9y ago
    Recently I was buying some gym clothes and wanted to buy Nike. I stopped to think about why, and traced it back to the early 2000s when I got into golfing, and Tiger Woods was a big influence for me. At that time I bought all Nike clothing. After realizing that I just need some tops and shorts to sweat in at home, I ended up buying the store brand label, saving myself ~80% in costs. It's funny how the mind works.
- michaelt 9y ago
  I do not understand why it’s hard to see at least the end result. The longer it takes for a change in input to produce a change in output, the greater the chance confounding factors will interfere, making your conclusions less convincing. You moved your spending from billboards to print magazines, and 6 months later sales are higher. How do you tell if it was the ads, when there was also a model refresh, your competitor released a poorly reviewed product, and the economy has been kind to your target demographic? As John Wanamaker supposedly said, "Half the money I spend on advertising is wasted; the trouble is I don't know which half"
- shostack 9y ago
  Others have done a great job of highlighting some of the challenges, but I wanted to share my own thoughts. Conducting an on/off test can be a viable approach in some cases assuming you can properly control for the numerous (understatement) variables that often make your data incredible dirty for such a test. That said, often the nature and number of the variables is such that you would not get meaningful results from an on/off test. This is quite often the case with large brands. As such, other forms of testing are often used individually or in conjunction with each other to get a better read on things. In terms of the end-to-end data, from an online advertising standpoint you often are missing trackable data points around other brand touch points the person might have encountered before purchase, or often, even the end purchase itself. And that's just the start of the data headaches.
tyfon 9y ago
There is also absolutely no will in the marketing departments I've seen to measure negative effects of advertising, only the positive. Myself I have an adverse effect when I see an ad for a product. I will actively try to avoid it as I feel if you have to advertise, the competition probably has a superior product.
- mcny 9y ago
  I completely agree. I feel this way for things that I think should not have to advertise. For instance, higher education. I saw a poster at Forest Hills 71 Avenue subway station for Queens College and now I feel like it is not a top class college. I think my reaction would have been different of the ad was subtler though. Thoughts?
  - magic_beans 9y ago
    Ads for colleges in the subway aren’t meant for you, though. You probably already have a college degree and are well-educated. Those subway ads are for people who aren’t as well-educated, never went to college and are trying to fit it in between working, and wouldn’t get into a top-tier college.
    - ghaff 9y ago
      Exactly. The top-tier college in the city probably has a subway stop with its name in it. Those ads are aimed at someone who doesn't have a college degree and/or has been contemplating doing something positive about their career. So part-time/night college, nursing school, etc. ads are there to nudge them and to suggest a specific option for them to consider.
    - xaqfox 9y ago
      I believe the thought was that if they were so prestigious they would not need to advertise at all. I have never felt this way about Universities, but I certainly have for defense attorneys.
      - 9y ago

mkagenius 9y ago

I do not understand why it’s hard to see at least the end result. Before switching on the ads and after switching on the ads. I am sure a company as big as proctor and gamble has at least a team of people who are dedicated to gather the end to end metric. Then where’s the gap?

ACow_Adonis 9y ago
One big part of the problem is whether that team (if they exist at all, and that's a big IF) works for the marketing department and whether they have an inherent conflict of interest. In my experience as a data scientist working along-side some marketing people, most of the performance measuring, if it was ever done, was total psuedo-science. It was almost universally done either by people aligned with the actual marketing or who actually worked on said marketing effort. Not only is there the universal mantra of "Don't ever come to a finding which says that your team and effort fails to contribute value to the company", but lets say that you actually are a data scientist and you really are independent and you come to a conclusion that your multi-million dollar marketing campaign is doing sweet f all. This will generally be despite 6 months of presentations from the marketing team about how great it was, and probably several bonuses/pay reviews. And now you're not only fighting a civil war in your company, you're doing it against multiple people with millions of dollars in budget, who have been proclaiming their success, and who's entire career has been based upon appealing to peoples guts to give them high paying jobs and big budgets. Good luck with that fight... Rule 37 of professional analytics in big companies: be VERY careful before turning your analytical eye inwards into your own companies' hierarchy/practices...
- wastedhours 9y ago
  I think we're reaching that crux point now - it's always been a black box of advertising online, but now teams are taking some knowledge in-house, there's increasing accountability, and thus cutting of budgets as they're clocking onto ROI (which a lot of channels can't show).
matt4077 9y ago
Your decision to buy Nike sneakers might be influenced by Nike's sponsorship of Uruguay in the 1998 World Cup. Maybe you were a child back then, and thought their goalkeeper was just the coolest dude, ever. See where the problem might be?
- nekopa 9y ago
  Recently I was buying some gym clothes and wanted to buy Nike. I stopped to think about why, and traced it back to the early 2000s when I got into golfing, and Tiger Woods was a big influence for me. At that time I bought all Nike clothing. After realizing that I just need some tops and shorts to sweat in at home, I ended up buying the store brand label, saving myself ~80% in costs. It's funny how the mind works.
michaelt 9y ago
I do not understand why it’s hard to see at least the end result. The longer it takes for a change in input to produce a change in output, the greater the chance confounding factors will interfere, making your conclusions less convincing. You moved your spending from billboards to print magazines, and 6 months later sales are higher. How do you tell if it was the ads, when there was also a model refresh, your competitor released a poorly reviewed product, and the economy has been kind to your target demographic? As John Wanamaker supposedly said, "Half the money I spend on advertising is wasted; the trouble is I don't know which half"
shostack 9y ago
Others have done a great job of highlighting some of the challenges, but I wanted to share my own thoughts. Conducting an on/off test can be a viable approach in some cases assuming you can properly control for the numerous (understatement) variables that often make your data incredible dirty for such a test. That said, often the nature and number of the variables is such that you would not get meaningful results from an on/off test. This is quite often the case with large brands. As such, other forms of testing are often used individually or in conjunction with each other to get a better read on things. In terms of the end-to-end data, from an online advertising standpoint you often are missing trackable data points around other brand touch points the person might have encountered before purchase, or often, even the end purchase itself. And that's just the start of the data headaches.

tyfon 9y ago

There is also absolutely no will in the marketing departments I've seen to measure negative effects of advertising, only the positive. Myself I have an adverse effect when I see an ad for a product. I will actively try to avoid it as I feel if you have to advertise, the competition probably has a superior product.

mcny 9y ago
I completely agree. I feel this way for things that I think should not have to advertise. For instance, higher education. I saw a poster at Forest Hills 71 Avenue subway station for Queens College and now I feel like it is not a top class college. I think my reaction would have been different of the ad was subtler though. Thoughts?
- magic_beans 9y ago
  Ads for colleges in the subway aren’t meant for you, though. You probably already have a college degree and are well-educated. Those subway ads are for people who aren’t as well-educated, never went to college and are trying to fit it in between working, and wouldn’t get into a top-tier college.
  - ghaff 9y ago
    Exactly. The top-tier college in the city probably has a subway stop with its name in it. Those ads are aimed at someone who doesn't have a college degree and/or has been contemplating doing something positive about their career. So part-time/night college, nursing school, etc. ads are there to nudge them and to suggest a specific option for them to consider.
  - xaqfox 9y ago
    I believe the thought was that if they were so prestigious they would not need to advertise at all. I have never felt this way about Universities, but I certainly have for defense attorneys.
    - 9y ago