3 ms·
I hypothesize without much direct experience that the executives of an organization should come to see excessively optimized metrics as a red flag. If what you
by jerf 4y ago
I hypothesize without much direct experience that the executives of an organization should come to see excessively optimized metrics as a red flag. If what you are measuring has been optimized to the n'th degree that is sufficient proof that something you are not measuring is suffering. There is always something important you are not measuring. It may be something measurable you aren't measuring today, or it may be an unmeasurable, but there's something.
Creating a new metric and seeing it go up for a year or two is good. Creating a new metric and seeing it go up and up and up past that ought to be considered suspicious. Not automatically "wrong", but something to dig into, rather than celebrate on cognitive autopilot.
- compiler-guy 4y agoThere are also diminishing marginal returns of optimizing for a certain metric. The first big steps are relatively easy, and produce a good return on the investment, but as you get deeper into the optimization, the costs of getting that next little bit go up. And people are tempted to cheat, and take short cuts to meet the goal. For example, getting from one nine of uptime to two nines isn't all that hard or expensive (a month of downtime per year to half a week). Getting from three nines to four nines is a lot harder, and a lot more expensive, and if you tie success at the organization to that improvement, people will fudge to reach it. Nearly every metric is like that. Amount of sales, tail latency, recruiting efficacy.