7 ms·

What if performance advertising isn't just an analytics scam?

86 points by mkmk 5y ago

nyxaiur 5y ago
No Performancer Marketer takes it for face value, we all know they don't, the problem is they still sell it to their clients as if they did.
- jamiequint 5y ago
  None of the good agencies do this.
  - nyxaiur 5y ago
    Well if you can give us a list of "good" agencies we will all be better of I guess. Thanks in advance.
    - franczesko 5y ago
      Wouldn't this apply to any external services out there? Agencies, software houses, etc. I'm not defending agencies (not glorifying them either), but it's hard to expect from the business model which scales costs proportionally to people hired and not the output, to be lean. In the end of the day, they need to keep the lights on and it's not their money anyway.
      - nyxaiur 5y ago
        In the end we all do and we can decide if we do it with fraud or not.
  - jpdaigle 5y ago
    Isn't that basically a "no true Scotsman" fallacy, though? I can completely believe both sides of this: the original Fishkin article, and this rebuttal which claims that "actually, marketers are competent and knowledgeable". One way to look at this, mindful of [Sturgeon's Law](https://en.wikipedia.org/wiki/Sturgeon%27s_law https://en.wikipedia.org/wiki/Sturgeon%27s_law) stating that ninety percent of everything is crap, is that good marketing agencies will rise above, but if I own a store selling scented candles and Google "online advertising" and hire one of the random agencies listed, maybe, just maybe, I'll get one of the 90% of bad ones that will try to hoodwink me into believing unsupported incremental conversion numbers.
    - dontich 5y ago
      Personally I have never found a good agency for these type of things, and I have worked with a bunch. Some in the gaming space were decent enough.
      - franczesko 5y ago
        The truth is that with performance marketing the know-how stays in-house. Agencies can't compete with r&d and dedicated analytical and engineering resources big brands have. The services offered are "for the rest of us".
      - soared 5y ago
        From my experience agencies around 200 employees that specialize seem to be the best. “Full service” means shitty at everything. Too small of an agency and you don’t get experts (or the experts leave), too big and you’re stuck in bureaucracy, huge margins, occasion very under skilled employees who go unnoticed, etc. I’m also starting to see more often the game brands play if switching vendors constantly so that each vendor operates at a loss to try and win the clients business. But the client always leaves by design.
PragmaticPulp 5y ago
> As impressive as these examples might seem, we have to consider the fact that no company would ever publicise their results had they observed the opposite. No CMO is going to go on Medium and write an article called We dropped our spend by X%, and saw our volume drop by X%. Even if they did write such a case study, it's unimaginable that it should become part of modern marketing folklore in the way that Airbnb and Ebay's experiments have In other words: Survivorship bias is the driver of these anecdotes. Obviously no company is going to brag publicly about shooting themselves in the foot by reducing advertising spend, so the only possible anecdotes that can exist publicly are positive ones. Also: Most companies aren't established, household brand names like AirBnB or Uber or eBay. Those companies have built enough of a reputation and have enough word of mouth momentum that advertising campaigns aren't going to move the needle much. I agree with much of this article. The original claims that performance advertising is a "scam" is playing to general distrust of marketers or a dislike of advertising. Any small companies trying to ditch advertising because AirBnB did it (after they were established and had saturated the market) are making their decisions for the wrong reasons.
- giaour 5y ago
  I don’t understand the survivorship bias argument. If any company tried to follow in AirBnB’s footsteps and saw their revenue decline as ad spend went down, wouldn’t they widely publicize this fact? “Hey! AirBnB is wrong, and following their advice nearly destroyed my small business” would be a compelling read and get a lot of traction. I find it hard to believe that such a demonstration of the value of marketing wouldn’t be immediately shouted from the rooftops by CMOs everywhere and find its way into marketing textbooks.
  - eatonphil 5y ago
    I agree. People publish post-mortems all the time. I don't think it's obvious good companies/organizations would be incentivized to keep a negative learning experience hidden.
  - mattnewton 5y ago
    Of course people would be hesitant to widely publicize a failure of your strategy like that.
dontich 5y ago
On the Airbnb point -- we ran many incrementality experiments prior to COVID -- COVID just completely changes things. Also 5% of Airbnb's revenue is alot.
notreallyhere00 5y ago
Wow. This article is excellent. It’s not just the content; he’s even made nice graphics… And the aesthetic of the post on mobile is 10/10. The issues at hand in this post are real. 1. Problem 1 - Awareness Many marketers outside of mega super super high paying startups are BARELY aware of these factors. I’m an amateur at this stuff but people look at me like I’m a wizard when I get my - VERY SIMPLE - Spreadsheets out. That’s one thing. Problem 2 - The tools these platforms and analytics tools are kind of shit at piecing the story together. I’m the sole marketer at a (very rapidly growing, if I may) software startup. Every so often, as an exercise, I go back and piece together the entire journey of our highest value opportunities and prepare a nice little report. The idea is to show the team that the buying process is complex and that tracking in our neat little funnels is a rough proxy at best. Problem 3 - untracked interaction We sell a product that is used by teams. This means that a TON of our traffic is going to be from team members coming in to have a look after team member A discovers us. But what happens if it’s team member C that gets in touch? How do you attribute that? Doing regular deep dives on individual customers is the best way to maintain sanity in an organisation and stop the inane conversations that non-marketing team members tend to start. The deep dives, in my experience, tend to be the thing that most generates trust in the marketing activity - because it goes from being just an abstract game of big numbers to ‘oh hey, that actually works, how clever.’
- franczesko 5y ago
  In a smaller scale this probably would be very effective, however I can't see this being possible with bigger client volumes. Also, I noticed in my career some sort of elitism and looking at the marketing from above, which is driven by ignorance in most of the cases.
  - notreallyhere00 5y ago
    It’s not possible to do this with all clients; that’s not my argument. But in a big company it should be done with as a matter of practise with at least a cohort. But yes there is a ton of ignorance in this world. I constantly have to remind folks that the world outside of Spreadheets is insanely messy.
ericffr 5y ago
I found Fishkin article more convincing that this one. The author seems to try to justify his worth (and job) rather than understanding the ads market.
howolduis 5y ago
I thought this article would be about Viagra ads
jfoster 5y ago
One point that I'm surprised the article didn't make: despite the findings of their experiments, Airbnb & Ebay still advertise.
- bryans 5y ago
  Not only do they both still advertise, but their ability to maintain revenue without massive ad spending is based entirely on their users marketing on their behalf -- not on some imaginary perpetual momentum. At least half of Airbnb hosts have their own website and many make their own local and regional ad buys.

86 points by mkmk 5y ago

No Performancer Marketer takes it for face value, we all know they don't, the problem is they still sell it to their clients as if they did.

jamiequint 5y ago
None of the good agencies do this.
- nyxaiur 5y ago
  Well if you can give us a list of "good" agencies we will all be better of I guess. Thanks in advance.
  - franczesko 5y ago
    Wouldn't this apply to any external services out there? Agencies, software houses, etc. I'm not defending agencies (not glorifying them either), but it's hard to expect from the business model which scales costs proportionally to people hired and not the output, to be lean. In the end of the day, they need to keep the lights on and it's not their money anyway.
    - nyxaiur 5y ago
      In the end we all do and we can decide if we do it with fraud or not.
- jpdaigle 5y ago
  Isn't that basically a "no true Scotsman" fallacy, though? I can completely believe both sides of this: the original Fishkin article, and this rebuttal which claims that "actually, marketers are competent and knowledgeable". One way to look at this, mindful of [Sturgeon's Law](https://en.wikipedia.org/wiki/Sturgeon%27s_law https://en.wikipedia.org/wiki/Sturgeon%27s_law) stating that ninety percent of everything is crap, is that good marketing agencies will rise above, but if I own a store selling scented candles and Google "online advertising" and hire one of the random agencies listed, maybe, just maybe, I'll get one of the 90% of bad ones that will try to hoodwink me into believing unsupported incremental conversion numbers.
  - dontich 5y ago
    Personally I have never found a good agency for these type of things, and I have worked with a bunch. Some in the gaming space were decent enough.
    - franczesko 5y ago
      The truth is that with performance marketing the know-how stays in-house. Agencies can't compete with r&d and dedicated analytical and engineering resources big brands have. The services offered are "for the rest of us".
    - soared 5y ago
      From my experience agencies around 200 employees that specialize seem to be the best. “Full service” means shitty at everything. Too small of an agency and you don’t get experts (or the experts leave), too big and you’re stuck in bureaucracy, huge margins, occasion very under skilled employees who go unnoticed, etc. I’m also starting to see more often the game brands play if switching vendors constantly so that each vendor operates at a loss to try and win the clients business. But the client always leaves by design.

PragmaticPulp 5y ago

> As impressive as these examples might seem, we have to consider the fact that no company would ever publicise their results had they observed the opposite. No CMO is going to go on Medium and write an article called We dropped our spend by X%, and saw our volume drop by X%. Even if they did write such a case study, it's unimaginable that it should become part of modern marketing folklore in the way that Airbnb and Ebay's experiments have In other words: Survivorship bias is the driver of these anecdotes. Obviously no company is going to brag publicly about shooting themselves in the foot by reducing advertising spend, so the only possible anecdotes that can exist publicly are positive ones. Also: Most companies aren't established, household brand names like AirBnB or Uber or eBay. Those companies have built enough of a reputation and have enough word of mouth momentum that advertising campaigns aren't going to move the needle much. I agree with much of this article. The original claims that performance advertising is a "scam" is playing to general distrust of marketers or a dislike of advertising. Any small companies trying to ditch advertising because AirBnB did it (after they were established and had saturated the market) are making their decisions for the wrong reasons.

giaour 5y ago
I don’t understand the survivorship bias argument. If any company tried to follow in AirBnB’s footsteps and saw their revenue decline as ad spend went down, wouldn’t they widely publicize this fact? “Hey! AirBnB is wrong, and following their advice nearly destroyed my small business” would be a compelling read and get a lot of traction. I find it hard to believe that such a demonstration of the value of marketing wouldn’t be immediately shouted from the rooftops by CMOs everywhere and find its way into marketing textbooks.
- eatonphil 5y ago
  I agree. People publish post-mortems all the time. I don't think it's obvious good companies/organizations would be incentivized to keep a negative learning experience hidden.
- mattnewton 5y ago
  Of course people would be hesitant to widely publicize a failure of your strategy like that.

dontich 5y ago

On the Airbnb point -- we ran many incrementality experiments prior to COVID -- COVID just completely changes things. Also 5% of Airbnb's revenue is alot.

notreallyhere00 5y ago

Wow. This article is excellent. It’s not just the content; he’s even made nice graphics… And the aesthetic of the post on mobile is 10/10. The issues at hand in this post are real. 1. Problem 1 - Awareness Many marketers outside of mega super super high paying startups are BARELY aware of these factors. I’m an amateur at this stuff but people look at me like I’m a wizard when I get my - VERY SIMPLE - Spreadsheets out. That’s one thing. Problem 2 - The tools these platforms and analytics tools are kind of shit at piecing the story together. I’m the sole marketer at a (very rapidly growing, if I may) software startup. Every so often, as an exercise, I go back and piece together the entire journey of our highest value opportunities and prepare a nice little report. The idea is to show the team that the buying process is complex and that tracking in our neat little funnels is a rough proxy at best. Problem 3 - untracked interaction We sell a product that is used by teams. This means that a TON of our traffic is going to be from team members coming in to have a look after team member A discovers us. But what happens if it’s team member C that gets in touch? How do you attribute that? Doing regular deep dives on individual customers is the best way to maintain sanity in an organisation and stop the inane conversations that non-marketing team members tend to start. The deep dives, in my experience, tend to be the thing that most generates trust in the marketing activity - because it goes from being just an abstract game of big numbers to ‘oh hey, that actually works, how clever.’

franczesko 5y ago
In a smaller scale this probably would be very effective, however I can't see this being possible with bigger client volumes. Also, I noticed in my career some sort of elitism and looking at the marketing from above, which is driven by ignorance in most of the cases.
- notreallyhere00 5y ago
  It’s not possible to do this with all clients; that’s not my argument. But in a big company it should be done with as a matter of practise with at least a cohort. But yes there is a ton of ignorance in this world. I constantly have to remind folks that the world outside of Spreadheets is insanely messy.

ericffr 5y ago

I found Fishkin article more convincing that this one. The author seems to try to justify his worth (and job) rather than understanding the ads market.

howolduis 5y ago

I thought this article would be about Viagra ads

jfoster 5y ago

One point that I'm surprised the article didn't make: despite the findings of their experiments, Airbnb & Ebay still advertise.

bryans 5y ago
Not only do they both still advertise, but their ability to maintain revenue without massive ad spending is based entirely on their users marketing on their behalf -- not on some imaginary perpetual momentum. At least half of Airbnb hosts have their own website and many make their own local and regional ad buys.