3 ms·

I do not understand why it’s hard to see at least the end result. Before switching on the ads and after switching on the ads. I am sure a company as big as pro

I do not understand why it’s hard to see at least the end result. Before switching on the ads and after switching on the ads. I am sure a company as big as proctor and gamble has at least a team of people who are dedicated to gather the end to end metric. Then where’s the gap?

ACow_Adonis 9y ago
One big part of the problem is whether that team (if they exist at all, and that's a big IF) works for the marketing department and whether they have an inherent conflict of interest. In my experience as a data scientist working along-side some marketing people, most of the performance measuring, if it was ever done, was total psuedo-science. It was almost universally done either by people aligned with the actual marketing or who actually worked on said marketing effort. Not only is there the universal mantra of "Don't ever come to a finding which says that your team and effort fails to contribute value to the company", but lets say that you actually are a data scientist and you really are independent and you come to a conclusion that your multi-million dollar marketing campaign is doing sweet f all. This will generally be despite 6 months of presentations from the marketing team about how great it was, and probably several bonuses/pay reviews. And now you're not only fighting a civil war in your company, you're doing it against multiple people with millions of dollars in budget, who have been proclaiming their success, and who's entire career has been based upon appealing to peoples guts to give them high paying jobs and big budgets. Good luck with that fight... Rule 37 of professional analytics in big companies: be VERY careful before turning your analytical eye inwards into your own companies' hierarchy/practices...
- wastedhours 9y ago
  I think we're reaching that crux point now - it's always been a black box of advertising online, but now teams are taking some knowledge in-house, there's increasing accountability, and thus cutting of budgets as they're clocking onto ROI (which a lot of channels can't show).
matt4077 9y ago
Your decision to buy Nike sneakers might be influenced by Nike's sponsorship of Uruguay in the 1998 World Cup. Maybe you were a child back then, and thought their goalkeeper was just the coolest dude, ever. See where the problem might be?
- nekopa 9y ago
  Recently I was buying some gym clothes and wanted to buy Nike. I stopped to think about why, and traced it back to the early 2000s when I got into golfing, and Tiger Woods was a big influence for me. At that time I bought all Nike clothing. After realizing that I just need some tops and shorts to sweat in at home, I ended up buying the store brand label, saving myself ~80% in costs. It's funny how the mind works.
michaelt 9y ago
I do not understand why it’s hard to see at least the end result. The longer it takes for a change in input to produce a change in output, the greater the chance confounding factors will interfere, making your conclusions less convincing. You moved your spending from billboards to print magazines, and 6 months later sales are higher. How do you tell if it was the ads, when there was also a model refresh, your competitor released a poorly reviewed product, and the economy has been kind to your target demographic? As John Wanamaker supposedly said, "Half the money I spend on advertising is wasted; the trouble is I don't know which half"
shostack 9y ago
Others have done a great job of highlighting some of the challenges, but I wanted to share my own thoughts. Conducting an on/off test can be a viable approach in some cases assuming you can properly control for the numerous (understatement) variables that often make your data incredible dirty for such a test. That said, often the nature and number of the variables is such that you would not get meaningful results from an on/off test. This is quite often the case with large brands. As such, other forms of testing are often used individually or in conjunction with each other to get a better read on things. In terms of the end-to-end data, from an online advertising standpoint you often are missing trackable data points around other brand touch points the person might have encountered before purchase, or often, even the end purchase itself. And that's just the start of the data headaches.

ACow_Adonis 9y ago

One big part of the problem is whether that team (if they exist at all, and that's a big IF) works for the marketing department and whether they have an inherent conflict of interest. In my experience as a data scientist working along-side some marketing people, most of the performance measuring, if it was ever done, was total psuedo-science. It was almost universally done either by people aligned with the actual marketing or who actually worked on said marketing effort. Not only is there the universal mantra of "Don't ever come to a finding which says that your team and effort fails to contribute value to the company", but lets say that you actually are a data scientist and you really are independent and you come to a conclusion that your multi-million dollar marketing campaign is doing sweet f all. This will generally be despite 6 months of presentations from the marketing team about how great it was, and probably several bonuses/pay reviews. And now you're not only fighting a civil war in your company, you're doing it against multiple people with millions of dollars in budget, who have been proclaiming their success, and who's entire career has been based upon appealing to peoples guts to give them high paying jobs and big budgets. Good luck with that fight... Rule 37 of professional analytics in big companies: be VERY careful before turning your analytical eye inwards into your own companies' hierarchy/practices...

wastedhours 9y ago
I think we're reaching that crux point now - it's always been a black box of advertising online, but now teams are taking some knowledge in-house, there's increasing accountability, and thus cutting of budgets as they're clocking onto ROI (which a lot of channels can't show).

matt4077 9y ago

Your decision to buy Nike sneakers might be influenced by Nike's sponsorship of Uruguay in the 1998 World Cup. Maybe you were a child back then, and thought their goalkeeper was just the coolest dude, ever. See where the problem might be?

nekopa 9y ago
Recently I was buying some gym clothes and wanted to buy Nike. I stopped to think about why, and traced it back to the early 2000s when I got into golfing, and Tiger Woods was a big influence for me. At that time I bought all Nike clothing. After realizing that I just need some tops and shorts to sweat in at home, I ended up buying the store brand label, saving myself ~80% in costs. It's funny how the mind works.

michaelt 9y ago

I do not understand why it’s hard to see at least the end result. The longer it takes for a change in input to produce a change in output, the greater the chance confounding factors will interfere, making your conclusions less convincing. You moved your spending from billboards to print magazines, and 6 months later sales are higher. How do you tell if it was the ads, when there was also a model refresh, your competitor released a poorly reviewed product, and the economy has been kind to your target demographic? As John Wanamaker supposedly said, "Half the money I spend on advertising is wasted; the trouble is I don't know which half"

shostack 9y ago

Others have done a great job of highlighting some of the challenges, but I wanted to share my own thoughts. Conducting an on/off test can be a viable approach in some cases assuming you can properly control for the numerous (understatement) variables that often make your data incredible dirty for such a test. That said, often the nature and number of the variables is such that you would not get meaningful results from an on/off test. This is quite often the case with large brands. As such, other forms of testing are often used individually or in conjunction with each other to get a better read on things. In terms of the end-to-end data, from an online advertising standpoint you often are missing trackable data points around other brand touch points the person might have encountered before purchase, or often, even the end purchase itself. And that's just the start of the data headaches.