3 ms·

You have an unfortunate view that the ad attribution business is predicated on rational choices by advertisers. It’s more similar to investment management busi

by mlthoughts2018 6y ago

You have an unfortunate view that the ad attribution business is predicated on rational choices by advertisers. It’s more similar to investment management business. You could likewise say it’ll be easy for competitors to prove a firm is having bad quarters of portfolio underperformance, yet firms with bad track records consistently keep getting new business. Why is that? The underlying business proposition is more like consulting and blame protection. You can always fire your ad attribution partner and claim their “number crunching” was bogus to save your own ass. You don’t pay them to give you actionable ad strategy or performance measurement, just like eg the investment board of a pension fund doesn’t hire an investment manager to actually get higher returns. You pay them to make it look like you, the board member or the ad account manager, did all you could and any failings are down to uncontrollable poor performance of your vendor, which you then look like the good guy for firing, or look like the visionary for jumping ship to sexy AI solutions then jumping ship back to “classic” solutions when the AI solutions predictably don’t work. You do this for a while collecting bonuses and protecting your job, at the massive expense of the pension fund or advertising campaigns you are supposed to oversee, then you quit or get fired and play musical chairs to fail up into the same role at the next place. In many companies that run advertising, the ultimate choice of what ad spend to pursue is an unbelievably political shitstorm. There are arguments for spending on generic brand awareness (which has no goals related to attribution at all) vs ambitious “creative” ads & extravagance like Super Bowl ads, vs scientific ad experiments on defined cohorts of users. Nobody knows about (or gives a shit about) actually tracking the performance of all this beyond doing a bunch of headless chicken scrambling around to make some spreadsheets and slide decks to keep a job or argue for a bonus. To the executives a layer up it’s either a big cocktail party of all the different styles of pursuing ads and business is good and they don’t really care, or else it’s a needless cost center when business is bad and they give grain offerings to the advertising gods to decide what to cut. But nobody is seriously pursuing quant advertising. That is all just fluff branding. And this play by Apple is more of the same, just branding for professional services through using their anti-competitive moats around iOS.

tareqak 6y ago
I am confused. Advertising either works sufficiently well or it doesn’t. If it does work sufficiently well, then that means that it brings in more money than it costs. If targeted advertising works better than generic advertising, then targeted advertising brings in more than generic advertising. And vice versa. If advertising using privacy-protecting statistical methods works just as well, then again it should show up in the numbers.
- mlthoughts2018 6y ago
  You definitely don’t understand causal inference of ad attribution if this is what you think. I’ll give an example from a real experience I worked on before. A client was a large haircut chain and they gave us a data set of individuals that had received a specific ad campaign from them via a loyalty program (so it had a variety of demographic and customer history data). When we used the transactions data on this cohort to measure specific lift in the ad campaign, it was hugely negative - we thought there must be some kind of bug in the code, stressed out late nights coding and sanity checking everything and finding no issues that could explain it. Our internal managers said we cannot go back to the client and ask for details or anything until we fix the negative lift because “negative lift isn’t actually possible in real advertising.” (Literal quote from an experienced manager in a well-known advertising analytics firm - I even showed him plenty of research out of Facebook, Google and academia disproving it, but it was totally taboo to mention negative lift). Finally after some agonizing weeks we had no choice but to go back to the client and ask them to explain the full mechanics of their ad delivery, how people were targeted, to see if there was some bias or something we had no way to account for. And of course - they told us all these individuals had been given a physical coupon as they were leaving from a haircut. OMG! The physical mechanics of a haircut means you are obviously not coming back for another one for say 3-4 weeks on average, and you also won’t have perfect customer retention, so basically there is no way this kind of campaign can produce positive lift unless you build a statistical model that successfully controls for the built in customer churn and the mechanical delay period before the ad treatment could possibly causally effect people’s next haircut choice. This is just a drop in the bucket of strange things advertisers do. Often they won’t tell you targeting details like this, because they believe their method of targeting (even as simple as a coupon for loyalty rewards members) is some type of proprietary technique. This gets really really bad with firms that run their own digital advertising. If they are targeting “upper income people who just bought a BMW” via The Trade Desk, that’ll hugely skew any attribution modeling if they refuse to tell you they did that. A ton of the quant work of ad attribution is trying to automatically control for as many of these glaring omissions from the client as you can, since they often lie, refuse to tell you, or are actually wrong about the targeting they believed they used. In short, measuring whether it was the ad’s effect that led to that macro increase or decrease in revenue is no easier than the general problem of causal inference from non-random trial data lacking clean splits, and often is much harder.

tareqak 6y ago

I am confused. Advertising either works sufficiently well or it doesn’t. If it does work sufficiently well, then that means that it brings in more money than it costs. If targeted advertising works better than generic advertising, then targeted advertising brings in more than generic advertising. And vice versa. If advertising using privacy-protecting statistical methods works just as well, then again it should show up in the numbers.

mlthoughts2018 6y ago
You definitely don’t understand causal inference of ad attribution if this is what you think. I’ll give an example from a real experience I worked on before. A client was a large haircut chain and they gave us a data set of individuals that had received a specific ad campaign from them via a loyalty program (so it had a variety of demographic and customer history data). When we used the transactions data on this cohort to measure specific lift in the ad campaign, it was hugely negative - we thought there must be some kind of bug in the code, stressed out late nights coding and sanity checking everything and finding no issues that could explain it. Our internal managers said we cannot go back to the client and ask for details or anything until we fix the negative lift because “negative lift isn’t actually possible in real advertising.” (Literal quote from an experienced manager in a well-known advertising analytics firm - I even showed him plenty of research out of Facebook, Google and academia disproving it, but it was totally taboo to mention negative lift). Finally after some agonizing weeks we had no choice but to go back to the client and ask them to explain the full mechanics of their ad delivery, how people were targeted, to see if there was some bias or something we had no way to account for. And of course - they told us all these individuals had been given a physical coupon as they were leaving from a haircut. OMG! The physical mechanics of a haircut means you are obviously not coming back for another one for say 3-4 weeks on average, and you also won’t have perfect customer retention, so basically there is no way this kind of campaign can produce positive lift unless you build a statistical model that successfully controls for the built in customer churn and the mechanical delay period before the ad treatment could possibly causally effect people’s next haircut choice. This is just a drop in the bucket of strange things advertisers do. Often they won’t tell you targeting details like this, because they believe their method of targeting (even as simple as a coupon for loyalty rewards members) is some type of proprietary technique. This gets really really bad with firms that run their own digital advertising. If they are targeting “upper income people who just bought a BMW” via The Trade Desk, that’ll hugely skew any attribution modeling if they refuse to tell you they did that. A ton of the quant work of ad attribution is trying to automatically control for as many of these glaring omissions from the client as you can, since they often lie, refuse to tell you, or are actually wrong about the targeting they believed they used. In short, measuring whether it was the ad’s effect that led to that macro increase or decrease in revenue is no easier than the general problem of causal inference from non-random trial data lacking clean splits, and often is much harder.