4 ms·

Sounds like a low-quality shop or one that isn't being paid to do the strategic thinking around their measurement to determine "if it works." As someone who is

Sounds like a low-quality shop or one that isn't being paid to do the strategic thinking around their measurement to determine "if it works." As someone who is deep in the thick of that stuff, I'll be the first to admit the data is very fuzzy, and relies on a lot of statistics to really get a sense of whether things are working when there is not a strong direct response component (see: brand display advertising). Even the value things like paid search are entirely subjective depending on the attribution model you use and how you measure incremental lift. But that isn't to say it is all voodoo that doesn't work. Its just that to measure it properly you need some very expensive technology and/or proper stats training.

TeMPOraL 11y ago
> But that isn't to say it is all voodoo that doesn't work. Its just that to measure it properly you need some very expensive technology and/or proper stats training. That's what I believe too. It does work when done properly, but most of the shops are not equipped to even begin to understand how to do it. What kind of expensive technology do you mean in particular, if I may ask?
- shostack 11y ago
  This is going down the whole attribution rabbit hole, which is arguably the most challenge issue in the advertising/marketing space today. Static attribution models like first touch, last touch, U-shaped, and even custom weighted models provided multiple lenses through which to view your data to see how it would be valued if more credit was given towards it (since historically things were viewed via last touch lenses for the most part). Google Analytics offers some great attribution tools for free if you're interested in checking those out. Beyond that, static models have their weaknesses, so dynamic models such as the data-driven model in Google Analytics Premium are a nice step up. There are other solutions that handle this too such as VisualIQ, Adometry, Convertro, etc. Those were all recently acquired by Google, and others btw. Then there is software that attempts to help you model incremental lift in addition to dynamic attribution models. I just demoed Adobe's econometric modeling tool in their Adobe Analytics package (that feature set might be in the higher-level version) and that was about the closest I've seen to date for doing it the way I consider the "right" way (ie. user-path level attribution and modeling for incremental lift using literally whatever external data you can feed into it). I'm an ad tech nerd so let me know if you have any other questions.
  - walterbell 11y ago
    Do you know of any tools/studies/approaches which combine techniques from adtech attribution with techniques from online training, e.g. spaced repetition? E.g. are there timing models for saturation thresholds which cause users to ignore a message?
    - shostack 11y ago
      Are you getting things like modeling "banner blindness" and the reduction (or increase) in value of a given creative/channel based on increasing or decreasing the frequency cap? I haven't seen anything really that handles the reporting of that specifically. Typically modeling ideal frequency caps is left as an exercise to the user, and banner blindness is typically looked at by CTR as certain channels like FB can see a steep drop in that pretty quickly after as little as 2 weeks if you have a relatively small audience you are saturating. This gets more interesting on the retargeting side as people often complain about constantly seeing ads for a company--yet the reality is higher frequency caps often work because it keeps the brand top of mind or reminds them they needed to do something. But again, not familiar with any specific technical solutions with features built around modeling that and providing recommendations. That is still largely a human effort that requires analysis of touch point data on the frequency cap side, and other directional metrics on the banner blindness side. Would love to know if you or others know of any solutions that handle that better. I'm sure DSPs/networks like Criteo and others have their own in-house solutions, but part of the challenge is that there's no great way to set global frequency caps across DSPs/networks if you use multiple platforms.
fredkbloggs 11y ago
It might help to be clearer about what I'm getting at here. The question is not whether effective (and cost-effective) advertising exists, nor whether anyone anywhere knows how effective their advertising is. Clearly it does and some people do. It's whether (a) buying ad impressions on a clickbait listicle (or other "bullshit" page) is cost-effective or effective at all, (b) whether writing and hosting such pages is profitable once all internal costs are properly considered, and (c) whether the advertising formats the author is complaining about are cost-optimal or even effective at all. I took the GP's claim that "it works" to mean that the answers to all three questions are an emphatic yes. I'm not so sure we know that, and I'm not so sure it's true. It seems likely that the answers might well be "almost never", "probably not", and "maybe sometimes", but factors other than profitability are at work that cause the people making the decisions to do these things even when an omniscient rational corporation would not. Improper use of statistics or poor data collection could certainly be an explanation for that, but I propose that it's more basic than that. Of course, I might also be wrong, and maybe all this bullshit does work, and works very well. That just doesn't seem like the slam dunk that was implied.
- shostack 11y ago
  Ok, let's get specific about listicle crap like this then (aka native advertising). The formats, popularized by such ad networks as Taboola and Outbrain gained popularity because publishers needed ways to drive additional ad revenue without adding what appeared to be more display ads. This happened to combine with the layout/design tactics of sites like Buzzfeed and HuffPo that have nailed the approach to keeping users engaged, clicking, and reading. Coincidentally, this also helped provide a source of content for sites didn't really care about drawing the line between journalism and sales. Not that more respected publications haven't been trying them--just with tighter controls over the advertisers (perhaps via direct sales only). Affiliates are all over these as expected, and if they can make it work, then that says something because affiliates operate on crap margins and have many other challenges. That said, there's almost certainly a significant amount of dollars in there from companies just testing it because you need to constantly be testing new traffic sources and approaches. In fact, I always recommend companies carve out a test budget that they can just consider wasted, and if it happens to perform or they learn something, then it usually pays off in spades. There are always those that are less savvy and tossing money away on wasted impressions. Then there are folks like myself who understand the limitations of the data, the nature of the medium, have their own audience data, etc. In many ways it parallels the differences between Wall St. and Main St. in the investing world.

TeMPOraL 11y ago

> But that isn't to say it is all voodoo that doesn't work. Its just that to measure it properly you need some very expensive technology and/or proper stats training. That's what I believe too. It does work when done properly, but most of the shops are not equipped to even begin to understand how to do it. What kind of expensive technology do you mean in particular, if I may ask?

shostack 11y ago
This is going down the whole attribution rabbit hole, which is arguably the most challenge issue in the advertising/marketing space today. Static attribution models like first touch, last touch, U-shaped, and even custom weighted models provided multiple lenses through which to view your data to see how it would be valued if more credit was given towards it (since historically things were viewed via last touch lenses for the most part). Google Analytics offers some great attribution tools for free if you're interested in checking those out. Beyond that, static models have their weaknesses, so dynamic models such as the data-driven model in Google Analytics Premium are a nice step up. There are other solutions that handle this too such as VisualIQ, Adometry, Convertro, etc. Those were all recently acquired by Google, and others btw. Then there is software that attempts to help you model incremental lift in addition to dynamic attribution models. I just demoed Adobe's econometric modeling tool in their Adobe Analytics package (that feature set might be in the higher-level version) and that was about the closest I've seen to date for doing it the way I consider the "right" way (ie. user-path level attribution and modeling for incremental lift using literally whatever external data you can feed into it). I'm an ad tech nerd so let me know if you have any other questions.
- walterbell 11y ago
  Do you know of any tools/studies/approaches which combine techniques from adtech attribution with techniques from online training, e.g. spaced repetition? E.g. are there timing models for saturation thresholds which cause users to ignore a message?
  - shostack 11y ago
    Are you getting things like modeling "banner blindness" and the reduction (or increase) in value of a given creative/channel based on increasing or decreasing the frequency cap? I haven't seen anything really that handles the reporting of that specifically. Typically modeling ideal frequency caps is left as an exercise to the user, and banner blindness is typically looked at by CTR as certain channels like FB can see a steep drop in that pretty quickly after as little as 2 weeks if you have a relatively small audience you are saturating. This gets more interesting on the retargeting side as people often complain about constantly seeing ads for a company--yet the reality is higher frequency caps often work because it keeps the brand top of mind or reminds them they needed to do something. But again, not familiar with any specific technical solutions with features built around modeling that and providing recommendations. That is still largely a human effort that requires analysis of touch point data on the frequency cap side, and other directional metrics on the banner blindness side. Would love to know if you or others know of any solutions that handle that better. I'm sure DSPs/networks like Criteo and others have their own in-house solutions, but part of the challenge is that there's no great way to set global frequency caps across DSPs/networks if you use multiple platforms.

fredkbloggs 11y ago

It might help to be clearer about what I'm getting at here. The question is not whether effective (and cost-effective) advertising exists, nor whether anyone anywhere knows how effective their advertising is. Clearly it does and some people do. It's whether (a) buying ad impressions on a clickbait listicle (or other "bullshit" page) is cost-effective or effective at all, (b) whether writing and hosting such pages is profitable once all internal costs are properly considered, and (c) whether the advertising formats the author is complaining about are cost-optimal or even effective at all. I took the GP's claim that "it works" to mean that the answers to all three questions are an emphatic yes. I'm not so sure we know that, and I'm not so sure it's true. It seems likely that the answers might well be "almost never", "probably not", and "maybe sometimes", but factors other than profitability are at work that cause the people making the decisions to do these things even when an omniscient rational corporation would not. Improper use of statistics or poor data collection could certainly be an explanation for that, but I propose that it's more basic than that. Of course, I might also be wrong, and maybe all this bullshit does work, and works very well. That just doesn't seem like the slam dunk that was implied.

shostack 11y ago
Ok, let's get specific about listicle crap like this then (aka native advertising). The formats, popularized by such ad networks as Taboola and Outbrain gained popularity because publishers needed ways to drive additional ad revenue without adding what appeared to be more display ads. This happened to combine with the layout/design tactics of sites like Buzzfeed and HuffPo that have nailed the approach to keeping users engaged, clicking, and reading. Coincidentally, this also helped provide a source of content for sites didn't really care about drawing the line between journalism and sales. Not that more respected publications haven't been trying them--just with tighter controls over the advertisers (perhaps via direct sales only). Affiliates are all over these as expected, and if they can make it work, then that says something because affiliates operate on crap margins and have many other challenges. That said, there's almost certainly a significant amount of dollars in there from companies just testing it because you need to constantly be testing new traffic sources and approaches. In fact, I always recommend companies carve out a test budget that they can just consider wasted, and if it happens to perform or they learn something, then it usually pays off in spades. There are always those that are less savvy and tossing money away on wasted impressions. Then there are folks like myself who understand the limitations of the data, the nature of the medium, have their own audience data, etc. In many ways it parallels the differences between Wall St. and Main St. in the investing world.