3 ms·

Surprised not to see Goodhart's Law[0] referenced here - "When a measure becomes a target, it ceases to be a good measure". Not the same concept, but a related

by scubbo 5y ago

Surprised not to see Goodhart's Law[0] referenced here - "When a measure becomes a target, it ceases to be a good measure". Not the same concept, but a related one (as is the Cobra Effect[1], of which the poppy-field burning is an example) [0] https://en.wikipedia.org/wiki/Goodhart's_law https://en.wikipedia.org/wiki/Goodhart's_law [1] https://en.wikipedia.org/wiki/Perverse_incentive#The_original_cobra_effect https://en.wikipedia.org/wiki/Perverse_incentive#The_origina...

aidenn0 5y ago
I also think that Goodhart's law is a stronger effect than the McNamara Fallacy for Afghanistan; when people are actively faking results, then that's not so much measuring the wrong things as just flat out failing at measuring things.
- austincheney 5y ago
  The most important line from the article: Those who are concerned about falling into the trap of the McNamara Fallacy shouldn’t abandon quantitative measurements and metrics. A key example is Moneyball: https://en.m.wikipedia.org/wiki/Moneyball https://en.m.wikipedia.org/wiki/Moneyball In my line of work, JavaScript, people will intentionally ignore data that challenges their favored approach. Measuring things is important, but all these examples illustrate that measures are irrelevant when ethics are absent, which is worse than measures applied too narrowly as in the McNamara Fallacy.
musicale 5y ago
From [1]: "It was discovered that, by providing company executives with bonuses for reporting higher earnings, executives at Fannie Mae and other large corporations were encouraged to artificially inflate earnings statements and make decisions targeting short-term gains at the expense of long-term profitability." What a shocking revelation! Could it possibly apply to other companies that report quarterly results? ;-)
- scubbo 5y ago
  I vividly remember a "small-hands" (smaller all-hands) very early in my career, where the VP basically said "we recognize that oncall is a burden, and we want to compensate you for it. At first we thought we should give you a bonus related to how many times you get paged - but then we realized that that's incentivizing you to build flaky systems. Then we considered giving a bonus in inverse relation to how often you're paged - but then we though, no, they're smart engineers, they'll just turn off the monitoring and collect the bonus. So we're giving you a flat rate". In the absence of some untamperable objective way to measure service health (the concept of SLAs was a distant dream at that point), can't fault that reasoning, tbh!
bambax 5y ago
In a documentary about the VietNam war (I think it's Ken Loach's), there's an army guy who says this: > When you can't count what's important, you make what you can count, important. That's the McNamara fallacy in a nutshell. Kill counts started to become the end-all be-all of the war because bodies are the easiest thing to count, but it wasn't the relevant metric. Tragedies are often the result of people persisting in something and failing to see the big picture. Even more so when numbers seem to tell them they're right. It doesn't matter that the numbers are right if we're looking at the wrong numbers.

by scubbo 5y ago

aidenn0 5y ago

I also think that Goodhart's law is a stronger effect than the McNamara Fallacy for Afghanistan; when people are actively faking results, then that's not so much measuring the wrong things as just flat out failing at measuring things.

austincheney 5y ago
The most important line from the article: Those who are concerned about falling into the trap of the McNamara Fallacy shouldn’t abandon quantitative measurements and metrics. A key example is Moneyball: https://en.m.wikipedia.org/wiki/Moneyball https://en.m.wikipedia.org/wiki/Moneyball In my line of work, JavaScript, people will intentionally ignore data that challenges their favored approach. Measuring things is important, but all these examples illustrate that measures are irrelevant when ethics are absent, which is worse than measures applied too narrowly as in the McNamara Fallacy.

musicale 5y ago

From [1]: "It was discovered that, by providing company executives with bonuses for reporting higher earnings, executives at Fannie Mae and other large corporations were encouraged to artificially inflate earnings statements and make decisions targeting short-term gains at the expense of long-term profitability." What a shocking revelation! Could it possibly apply to other companies that report quarterly results? ;-)

scubbo 5y ago
I vividly remember a "small-hands" (smaller all-hands) very early in my career, where the VP basically said "we recognize that oncall is a burden, and we want to compensate you for it. At first we thought we should give you a bonus related to how many times you get paged - but then we realized that that's incentivizing you to build flaky systems. Then we considered giving a bonus in inverse relation to how often you're paged - but then we though, no, they're smart engineers, they'll just turn off the monitoring and collect the bonus. So we're giving you a flat rate". In the absence of some untamperable objective way to measure service health (the concept of SLAs was a distant dream at that point), can't fault that reasoning, tbh!

bambax 5y ago

In a documentary about the VietNam war (I think it's Ken Loach's), there's an army guy who says this: > When you can't count what's important, you make what you can count, important. That's the McNamara fallacy in a nutshell. Kill counts started to become the end-all be-all of the war because bodies are the easiest thing to count, but it wasn't the relevant metric. Tragedies are often the result of people persisting in something and failing to see the big picture. Even more so when numbers seem to tell them they're right. It doesn't matter that the numbers are right if we're looking at the wrong numbers.