3 ms·

> As much as I dislike some of the current trends (and I do digital media for a living mind you), I do have to point out that this stuff wouldn't be done if it

> As much as I dislike some of the current trends (and I do digital media for a living mind you), I do have to point out that this stuff wouldn't be done if it didn't work. Are we sure of that? It might depend on our definition of "work". A lot of the people responsible for doing this stuff may just be doing it because if they admit that none of it works, they'll have nothing else to offer and will be out of a job. It's not terribly hard to make up data showing some kind of effectiveness, or just to misinterpret real data in a way that can convince others to let you keep doing it. For every marketing or advertising craftsman out there who builds something that works, gathers useful data, and interprets that data honestly, there are probably 20 people who have no clue and just keep churning out the same stuff because they don't have anything else to offer. Whether it's profitable, or profitable enough to be worth doing, is anyone's guess. Given how cheap impressions are, especially the kind of ultra-low-value impressions that bullshit provides, it's pretty difficult to imagine this stuff paying for itself, even if most of it is autogenerated or written by the world's most desperate English-speakers. It might help more to come up with other jobs for the people who are putting the bullshit out there. Jobs they can do well, ideally. As it stands, I'm not sure the bullshit is serving their employers as much as it's just something for them to do so they keep getting a paycheck.

TeMPOraL 11y ago
> It's not terribly hard to make up data showing some kind of effectiveness, or just to misinterpret real data in a way that can convince others to let you keep doing it. For every marketing or advertising craftsman out there who builds something that works, gathers useful data, and interprets that data honestly, there are probably 20 people who have no clue and just keep churning out the same stuff because they don't have anything else to offer. Let me share a personal anecdote from watching a social media marketing company at works. These marketing people got contracts to do brand/site/service promotion on everyone's favourite social networking and image collecting sites. They wrote many posts, made interesting graphics, etc. Generally did a good, honest work at that. Then every month they had to write reports detailing the effectiveness of their efforts. The way they did it was like this: 1) gather metrics provided by social media platforms they used 2) put them in a DOC file 3) start with the conclusion that they did their job well and helped the marketed company tremendously 4) invent some plausible-sounding reasons to connect the data from 1) to conclusion in 3) And mind you, this wasn't intentional dishonesty. They simply have no fucking clue about the meaning of the data, or what can (and can not) be reasonably inferred from it, so they don't even try. They don't even know what they don't know. But their clients don't know anything too; it's basically both sides not knowing what those colorful graphs and tables of numbers mean and bullshitting one another without even realizing it. I guess one can't generalize from one example, but mind this: social media marketing companies are not staffed by math PhDs, they're staffed by social sciences undergrads who decided that, given current state of the job market, learning Photoshop is better than working at McDonalds. They may be good at making interesting content, but they have zero necessary knowledge for getting right conclusions from data.
- shostack 11y ago
  Sounds like a low-quality shop or one that isn't being paid to do the strategic thinking around their measurement to determine "if it works." As someone who is deep in the thick of that stuff, I'll be the first to admit the data is very fuzzy, and relies on a lot of statistics to really get a sense of whether things are working when there is not a strong direct response component (see: brand display advertising). Even the value things like paid search are entirely subjective depending on the attribution model you use and how you measure incremental lift. But that isn't to say it is all voodoo that doesn't work. Its just that to measure it properly you need some very expensive technology and/or proper stats training.
  - TeMPOraL 11y ago
    > But that isn't to say it is all voodoo that doesn't work. Its just that to measure it properly you need some very expensive technology and/or proper stats training. That's what I believe too. It does work when done properly, but most of the shops are not equipped to even begin to understand how to do it. What kind of expensive technology do you mean in particular, if I may ask?
    - shostack 11y ago
      This is going down the whole attribution rabbit hole, which is arguably the most challenge issue in the advertising/marketing space today. Static attribution models like first touch, last touch, U-shaped, and even custom weighted models provided multiple lenses through which to view your data to see how it would be valued if more credit was given towards it (since historically things were viewed via last touch lenses for the most part). Google Analytics offers some great attribution tools for free if you're interested in checking those out. Beyond that, static models have their weaknesses, so dynamic models such as the data-driven model in Google Analytics Premium are a nice step up. There are other solutions that handle this too such as VisualIQ, Adometry, Convertro, etc. Those were all recently acquired by Google, and others btw. Then there is software that attempts to help you model incremental lift in addition to dynamic attribution models. I just demoed Adobe's econometric modeling tool in their Adobe Analytics package (that feature set might be in the higher-level version) and that was about the closest I've seen to date for doing it the way I consider the "right" way (ie. user-path level attribution and modeling for incremental lift using literally whatever external data you can feed into it). I'm an ad tech nerd so let me know if you have any other questions.

TeMPOraL 11y ago

> It's not terribly hard to make up data showing some kind of effectiveness, or just to misinterpret real data in a way that can convince others to let you keep doing it. For every marketing or advertising craftsman out there who builds something that works, gathers useful data, and interprets that data honestly, there are probably 20 people who have no clue and just keep churning out the same stuff because they don't have anything else to offer. Let me share a personal anecdote from watching a social media marketing company at works. These marketing people got contracts to do brand/site/service promotion on everyone's favourite social networking and image collecting sites. They wrote many posts, made interesting graphics, etc. Generally did a good, honest work at that. Then every month they had to write reports detailing the effectiveness of their efforts. The way they did it was like this: 1) gather metrics provided by social media platforms they used 2) put them in a DOC file 3) start with the conclusion that they did their job well and helped the marketed company tremendously 4) invent some plausible-sounding reasons to connect the data from 1) to conclusion in 3) And mind you, this wasn't intentional dishonesty. They simply have no fucking clue about the meaning of the data, or what can (and can not) be reasonably inferred from it, so they don't even try. They don't even know what they don't know. But their clients don't know anything too; it's basically both sides not knowing what those colorful graphs and tables of numbers mean and bullshitting one another without even realizing it. I guess one can't generalize from one example, but mind this: social media marketing companies are not staffed by math PhDs, they're staffed by social sciences undergrads who decided that, given current state of the job market, learning Photoshop is better than working at McDonalds. They may be good at making interesting content, but they have zero necessary knowledge for getting right conclusions from data.

shostack 11y ago
Sounds like a low-quality shop or one that isn't being paid to do the strategic thinking around their measurement to determine "if it works." As someone who is deep in the thick of that stuff, I'll be the first to admit the data is very fuzzy, and relies on a lot of statistics to really get a sense of whether things are working when there is not a strong direct response component (see: brand display advertising). Even the value things like paid search are entirely subjective depending on the attribution model you use and how you measure incremental lift. But that isn't to say it is all voodoo that doesn't work. Its just that to measure it properly you need some very expensive technology and/or proper stats training.
- TeMPOraL 11y ago
  > But that isn't to say it is all voodoo that doesn't work. Its just that to measure it properly you need some very expensive technology and/or proper stats training. That's what I believe too. It does work when done properly, but most of the shops are not equipped to even begin to understand how to do it. What kind of expensive technology do you mean in particular, if I may ask?
  - shostack 11y ago
    This is going down the whole attribution rabbit hole, which is arguably the most challenge issue in the advertising/marketing space today. Static attribution models like first touch, last touch, U-shaped, and even custom weighted models provided multiple lenses through which to view your data to see how it would be valued if more credit was given towards it (since historically things were viewed via last touch lenses for the most part). Google Analytics offers some great attribution tools for free if you're interested in checking those out. Beyond that, static models have their weaknesses, so dynamic models such as the data-driven model in Google Analytics Premium are a nice step up. There are other solutions that handle this too such as VisualIQ, Adometry, Convertro, etc. Those were all recently acquired by Google, and others btw. Then there is software that attempts to help you model incremental lift in addition to dynamic attribution models. I just demoed Adobe's econometric modeling tool in their Adobe Analytics package (that feature set might be in the higher-level version) and that was about the closest I've seen to date for doing it the way I consider the "right" way (ie. user-path level attribution and modeling for incremental lift using literally whatever external data you can feed into it). I'm an ad tech nerd so let me know if you have any other questions.