4 ms·
+1 Also, I feel like every system can be abused if the incentivised metric is defined poorly. Example: almost every advertising KPI. 1. A Brand wants to sell
by rpastuszak 7y ago
+1
Also, I feel like every system can be abused if the incentivised metric is defined poorly.
Example: almost every advertising KPI.
1. A Brand wants to sell a product (incentive: sales).
2. Agency sales people want to reach a particular KPI—budget in this case, the higher, the better.
3. An Ad Tech company will serve ads based on a KPI agreed with the Agency (e.g. CPM, impressions).
It's very easy to satisfy 2 and 3 without bringing any additional value to 1., for example by applying less strict viewability metrics (is 50% in view "visible" or 1%? should we pause the video ad below the fold? what is an impression, exactly?).
The original intent (sell more units of product x) is gone. This is due to systematic issues with the industry, not a single specific company. But mainly, this is due to the fact that the established incentives represent only a chunk of the reality and can be followed without focusing on the actual problem. The overlap between actors' goals is poorly defined, small and often doesn't exist.
In a nutshell: if we define a metric serving as a shortcut for our goal, people will find a way of improving the metric, but rarely reaching the goal. A broadly defined incentive is better than punishment.
- maccard 7y agoThat has a name, the Cobra Effect [0] [0] https://en.wikipedia.org/wiki/Cobra_effect https://en.wikipedia.org/wiki/Cobra_effect
- jefftk 7y agoThe Cobra Effect is when you're trying to make there be fewer of things (cobras) but the incentives you set up actually lead to there being more of them (it's now worth it for people to be cobra breeders). Perhaps you're thinking of Goodheart's law? https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law
- scarejunba 7y agoThat's why attribution is such a big story in advertising. With semi-persistent identifiers, you can learn the story of whether someone was: 1) Visited your store before they saw your ad 2) Only visited your store after seeing your ad You can do this online or for physical store locations. There are lots of players there because big marketing teams are fairly sophisticated now: they want cross-channel identity (across mobile, web, TV), attribution, and measurement.
- ec109685 7y agoA short term metric is necessary to allow campaigns to optimize their spend without a long multi-month turn around as conversions are fully tracked and users act upon the ad. While there is risk that the wrong short term metric is chosen (that is not indicative of the real goal the advertiser is trying to achieve), it is not an invalid approach. It is untrue that setting a short term goal will _rarely_ achieve the long term value. Otherwise, the advertiser will throw out that short-term goal since it is flawed.