3 ms·

It’s very dangerous to use a single metric as a proxy for something else that is obviously related to it but not directly so. We see this here with market share

It’s very dangerous to use a single metric as a proxy for something else that is obviously related to it but not directly so. We see this here with market share and profit share. We’ve seen similar things like lines of code and productivity. There’s an apocryphal story told about an IBM salesman. He was the sales leader year in and year out, selling almost as much as all the other fellows in his office combined. One day, his manager retires and Armonk sends out a hot shot MBA to run the sales team. The MBA summons the top salesman. “I’ve been running some numbers. Your sales are very good, but you’re only averaging 1.2 calls a day. The other guys are doing 3.7 calls. Imagine how much you could sell if you could get your average up to 3.7!” The sales guy realized he was going to have to train another manager. “Oh? I was wondering how much the other guys could sell if they did a better job of qualifying leads and got their averages down to 1.2..."

SatvikBeri 15y ago
Here's a very real, very dangerous example: conversion rate as a proxy for profit per visitor or # of unique people signing up. A web page that gets users to visit an average of 2 times before buying instead of an average of 3 times seems to have a vastly improved conversion rate (depending on how you measure), but is not improving sales one whit.
- coryl 15y ago
  I'm not sure the example you gave measures conversion at all. Isn't it measuring avg visits against a single fixed purchase?
  - SatvikBeri 15y ago
    A bad, very common (in my experience) definition of conversion rate is "goal actions/# of visits". It seems like you're thinking of conversion rate as "goal actions/# of unique visitors", which is a better definition, but still frequently at odds with profit.
Gustomaximus 15y ago
I think people tend to over analyse this stuff which is why bottom line numbers need to remain the focus (and so often don't). Anything before the bottom line figure is about understanding the result and aiding improvement but not defining the success of the activity. Also focusing on the final outcome avoids the distraction of people cherry picking statistic to make their campaigns look successful when they present results, which many people buy into unquestionably when they see a nice looking graph.

SatvikBeri 15y ago

Here's a very real, very dangerous example: conversion rate as a proxy for profit per visitor or # of unique people signing up. A web page that gets users to visit an average of 2 times before buying instead of an average of 3 times seems to have a vastly improved conversion rate (depending on how you measure), but is not improving sales one whit.

coryl 15y ago
I'm not sure the example you gave measures conversion at all. Isn't it measuring avg visits against a single fixed purchase?
- SatvikBeri 15y ago
  A bad, very common (in my experience) definition of conversion rate is "goal actions/# of visits". It seems like you're thinking of conversion rate as "goal actions/# of unique visitors", which is a better definition, but still frequently at odds with profit.

Gustomaximus 15y ago

I think people tend to over analyse this stuff which is why bottom line numbers need to remain the focus (and so often don't). Anything before the bottom line figure is about understanding the result and aiding improvement but not defining the success of the activity. Also focusing on the final outcome avoids the distraction of people cherry picking statistic to make their campaigns look successful when they present results, which many people buy into unquestionably when they see a nice looking graph.