5 ms·

Honestly though why tf did these companies never try lowering spend incrementally to see if it made a difference? Is this why engineers are so valued because we

by mlazos 6y ago

Honestly though why tf did these companies never try lowering spend incrementally to see if it made a difference? Is this why engineers are so valued because we understand the simple concept of AB testing?

Traster 6y ago
For the same reason that you never lower your margin. It's very easy to lower your margin, and you might get growth as a result of your lower margins, but it's a massive gamble with the most likely outcome being that you are now selling the same stuff for less money. And do you know what happens when you cut your marigns and don't see growth? You get fired. Same thing with advertising. You drop your ad spend, you lose market share, and you're fired. There is no step 4. If you're right and the ad spend was unnecessary, you've made a tiny difference (no one cares about cost, only growth), if you're wrong, you're fired.
texasbigdata 6y ago
Disclaimer: I’m not a marketing guy. It’s sold as hand wavy. I once dropped in on a consulting client (at the request of an investor on the board) and did a quick assessment. The assessment was holistic and marketing was one of many enterprise value levers, so did a rough check. The marketing firm, very similar to the Uber commentary in the underlying podcast above where he found 20,000 installs coming from an app with 30,000 MAUs, I calculated claimed performance which back of the envelope would imply you could capture the entire US vertical for that industry for sub $X0M of annual spend. I knew that space very well, and well that just was not possible. My quick question kicked off a large tap dance in which some finger pointing occurred and a “new attribution formula” was introduced. The vendor got fired. This was sort of a reasonably sized entity, with some sophistication. But think about it, unless you’re tech native, your VP of marketing has no granular data abilities in 2020, and unless you have a pool of SQL capable analysts lying around, AND you’re going to insist to validate on your own, it simply doesn’t work. So you trust the vendor. But wait the vendor gets paid X% of what you spend. Hmmm, would they realllllly work super hard to make sure the results are real if they are not easily found out to be falsified? This is a paranoid stance, obviously. But intrinsically structurally flawed. The Uber guy on the podcast straight up said he wouldn’t buy a dollar of programmatic ads without a big data team internally and a policy of (loosely quoted) “we need that file to check and if you don’t send it to us we’re cutting you off”. That’s just a lot of work.
reilly3000 6y ago
Marketers do tons of A/B testing and most measure conversions based on ad copy and placements. The problem often is they don’t look beyond aggregates ( as Google/Facebook flood marketers with dimensions, this one is easy to overlook). So it the overall ROI may be passable, but specific segments are over-analyzed (did YOU place the right message?) and others ignored. The platforms have never ever in any amount of years been willing to hand over raw clickstream logs of ad data, so marketers are forced to rely on misleading aggregates. I’m sick of engineer elitism. Try harder to see that brilliant folks exist in every department. You’ll be a happier person and learn a lot more. Marketing analytics is fascinating and full of puzzles. Marketers were driving engineers to make multivariate testing scale 20 years ago.

by mlazos 6y ago

Traster 6y ago

For the same reason that you never lower your margin. It's very easy to lower your margin, and you might get growth as a result of your lower margins, but it's a massive gamble with the most likely outcome being that you are now selling the same stuff for less money. And do you know what happens when you cut your marigns and don't see growth? You get fired. Same thing with advertising. You drop your ad spend, you lose market share, and you're fired. There is no step 4. If you're right and the ad spend was unnecessary, you've made a tiny difference (no one cares about cost, only growth), if you're wrong, you're fired.

texasbigdata 6y ago

Disclaimer: I’m not a marketing guy. It’s sold as hand wavy. I once dropped in on a consulting client (at the request of an investor on the board) and did a quick assessment. The assessment was holistic and marketing was one of many enterprise value levers, so did a rough check. The marketing firm, very similar to the Uber commentary in the underlying podcast above where he found 20,000 installs coming from an app with 30,000 MAUs, I calculated claimed performance which back of the envelope would imply you could capture the entire US vertical for that industry for sub $X0M of annual spend. I knew that space very well, and well that just was not possible. My quick question kicked off a large tap dance in which some finger pointing occurred and a “new attribution formula” was introduced. The vendor got fired. This was sort of a reasonably sized entity, with some sophistication. But think about it, unless you’re tech native, your VP of marketing has no granular data abilities in 2020, and unless you have a pool of SQL capable analysts lying around, AND you’re going to insist to validate on your own, it simply doesn’t work. So you trust the vendor. But wait the vendor gets paid X% of what you spend. Hmmm, would they realllllly work super hard to make sure the results are real if they are not easily found out to be falsified? This is a paranoid stance, obviously. But intrinsically structurally flawed. The Uber guy on the podcast straight up said he wouldn’t buy a dollar of programmatic ads without a big data team internally and a policy of (loosely quoted) “we need that file to check and if you don’t send it to us we’re cutting you off”. That’s just a lot of work.

reilly3000 6y ago

Marketers do tons of A/B testing and most measure conversions based on ad copy and placements. The problem often is they don’t look beyond aggregates ( as Google/Facebook flood marketers with dimensions, this one is easy to overlook). So it the overall ROI may be passable, but specific segments are over-analyzed (did YOU place the right message?) and others ignored. The platforms have never ever in any amount of years been willing to hand over raw clickstream logs of ad data, so marketers are forced to rely on misleading aggregates. I’m sick of engineer elitism. Try harder to see that brilliant folks exist in every department. You’ll be a happier person and learn a lot more. Marketing analytics is fascinating and full of puzzles. Marketers were driving engineers to make multivariate testing scale 20 years ago.