6 ms·

A closely-related effect I often cite is the McNamara fallacy[0], which is essentially about the tendency to focus on aspects of a system that are easily measur

A closely-related effect I often cite is the McNamara fallacy[0], which is essentially about the tendency to focus on aspects of a system that are easily measurable, often at the expense of aspects that are not. I see it as one of the weaknesses of the data-driven decision-making movement, since many interpret "data" to mean "numbers". I think this fallacy can partly explain why Goodhart's Law holds: it's the non-measurable (or difficult-to-measure) aspects that suffer most when a metric becomes a target, since measurable aspects could be (and often are) integrated into the target metric. [0]: https://en.wikipedia.org/wiki/McNamara_fallacy https://en.wikipedia.org/wiki/McNamara_fallacy

ghaff 5y ago
It's a genuinely hard problem because the quantifiable output metrics are easy to measure and an individual often does have some level of direct control over them. So we convince ourselves that they're a reasonable proxy for something we care about but aren't sure how to measure and that we have control systems in place, whether management or individual responsibility, that largely prevent e.g. quality being thrown away in pursuit of quantity. And we're often not entirely wrong if we do pick reasonable proxies and have reasonable control systems in place. Because throwing up our hands and saying metrics are useless is usually not the answer either.
- TeMPOraL 5y ago
  A flavor of this problem is what could be called "diffusion of responsibility", for a lack of better term. An individual who defined the measure and then optimizes for it will quickly figure out when their measure stops being a good proxy. But in organizations there usually isn't a single person who both understands what the measures are proxying, and has the power to remove a metric once it is used up, or get people to stop overfitting it.
  - Jtsummers 5y ago
    Another thing I've observed is when you have two measures that operate at different time scales. Both may even be valid measures, but the one measured (and responded to) more often has a stronger impact, and can negatively impact the less frequently measured metric when there's a conflict between them. A particular instance for this has been (to keep it simple) quantity (speed) and quality in production environments (factories and the like). Daily throughput measures paired with less frequent quality measures. The desire is to keep throughput high, and quality ends up suffering as a result. By integrating quality measures into the process you make the two measures compete on more equal footing, forcing a balance. At least one factory I worked in (well, adjacent to, I was in the software portion not the assembly line) massively reduced their quality problems by integrating quality checks between each station. This contrasted with the prior years where throughput, being measured and reacted to daily, drove them to make things so fast that they had piles of rework at the end. Integrating the quality measures between stations slowed them down, but their rework numbers turned into a rounding error (over a decade ago so I've forgotten the exact numbers, but they went from having items needing rework nearly every day to maybe one or two a month). As a result their real (deliverable to customers) production increased and their cost per unit dropped.
    - AceyMan 5y ago
      I see this as the root cause of the recently announced class action suit against LADWP over their implementation of tiered electricity pricing. The tiers (kwh rates) are in hunks of the a day measured in hours. But the reporting is only available to the consumer in the form of a monthly bill, so by the time you discover you were eating pixies in the Peak Cost hours the heat wave is over and your bill is already through the roof. (Any local SoCal residents please feel free to pick my analysis apart, but that was my first take when I heard about the legal action.)
- serverholic 5y ago
  The better option is to put someone in charge with a vision. However that is risky.
cryptica 5y ago
To make matters worse, it's also a vicious cycle. For example, if everyone is focused only on specific kinds of data and ignores observable reality, the trend in the data will become self-fulfilling until a point when the perceived inconsistency between the data and observable reality has grown so large that it becomes impossible to ignore. To understand what the big problems are today, you just have to think about the kinds of data which people in government (and the public) haven't been thinking about or aiming for. For example: Happiness, honesty, altruism, sanity... These are not measured and not targeted so they got completely crushed. In the past, large, powerful religious groups would target these characteristics but nowadays, society is more secular so these aspects of our lives have suffered significantly.
- serverholic 5y ago
  That's one of the thing Andrew Yang talks about. GDP isn't the only thing that matters, nor is it the best metric.
serverholic 5y ago
Another problem is that overly focusing on metrics means that people with good instincts are brought down to the same level as those with poor instincts. As long as metrics are increasing someone can write shit code and design a bloated product.
- BuyMyBitcoins 5y ago
  “Instinct” can’t be quantified and it’s such a shame that people with good track records can’t quite justify their choices quantitatively. This happens where I work constantly. The managers who get most of their projects delivered on time - I suspect because they have good intuition about effort levels - are constantly struggling to justify their higher initial estimates because the data driven people try to reduce development down to a formula.
burnafter182 5y ago
As it's told by Mandelbrot, the January effect was identified. Rather quickly it was exploited to the point where it no longer existed because the market exerted pressures to counteract it in the process of exploiting it. "Consider three cases. First, suppose a clever chart-reader thinks he has spotted a pattern in the old price records—say, every January, stock prices tend to rise. Can he get rich on that information, by buying in December and selling in January? Answer: No. If the market is big and efficient then others will spot the trend, too, or at least spot his trading on it. Soon, as more traders anticipate the January rally, more people are buying in December—and then, to beat the trend for a December rally, in November. Eventually, the whole phenomenon is spread out over so many months that it ceases to be noticeable. The trend has vanished, killed by its very discovery. In fact, in 1976 some economists spotted just such a pattern of regular January rallies in the stocks of small companies. Many investors close their losing positions towards the end of the year so they can book the loss as a tax deduction—and the market rebounds when they reinvest early in the new tax year. The effect is most pronounced on small stocks, which are more sensitive to small money movements. Alas, before you rush out to trade on this trend, you should know that its discovery seems to have killed it. After all the academic hoopla over it, it no longer shows up as clearly in price charts." -Benoit Mandelbrot, The Misbehavior of Markets https://en.wikipedia.org/wiki/January_effect https://en.wikipedia.org/wiki/January_effect
- miki123211 5y ago
  As far as I'm aware, this is why markets are considered a second-order chaotic system. In those systems, measurements of how the system performs can actually influence what happens next. This is in contrast to first order systems, i.e. the weather, which are hard to simulate, but the results of the simulations don't affect their accuracy.
- Stratoscope 5y ago
  This reminds me so much of a Christmas present hack my sister and I invented when we were kids. We used to open all our presents on Christmas morning. Then we begged "Can't we open a couple of presents on Christmas Eve?" So we got to open a few that night. Next year was "Well, how about Christmas Eve Morning? Maybe just one or two?" And the next year was "The 23rd is practically Christmas Eve, isn't it? It's just a few hours apart. Can't we open all our presents on the evening of the 23rd?" And we did! We didn't push it past that: we were already so happy that we got our presents a day and a half before all our friends!
rossdavidh 5y ago
In the context of manufacturing, W.E.Deming said something similar: "that which gets measured, gets improved". His conclusion from this was a little different than McNamara's, though. Since you will inevitably want to track your progress, make sure you track as many things as possible, because anything which is not tracked will get sacrificed to that which is. Up to a point, it's true. One issue is that some things, like vulnerability to supply chain disruptions, are intrinsically harder to track because they are based on rare occurrences. Thus, they will tend to get sacrificed in favor of measures which are more frequent, leading to an emphasis on short-term strategies.
- CalChris 5y ago
  Actually, Deming said much the opposite: It is wrong to suppose that if you can’t measure it, you can’t manage it – a costly myth. p. 26 of The New Economics for Industry, Government, Education It wasn't Drucker either. https://medium.com/centre-for-public-impact/what-gets-measured-gets-managed-its-wrong-and-drucker-never-said-it-fe95886d3df6 https://medium.com/centre-for-public-impact/what-gets-measur... This whole mindless must measure, must measure mentality has been criticized since the 50s. Measurement is a tool. There are many tools.
  - rossdavidh 5y ago
    In my experience, if it isn't measured, then it is assumed that the policy (whatever it is) is working. If you don't measure, you can't be surprised by "wow, it didn't work like we thought". Therefore, mistakes don't get recognized or corrected. There are many tools, but measurement is one of the only ones that brings unexpected bad news to the user, and that is invaluable.
  - mumblemumble 5y ago
    I would argue that it's worthwhile to measure as much as you can, insofar as it facilitates orderly decisionmaking processes. The problem is that people tend to think that all measurement is necessarily quantitative. I think that this might be a version of the streetlight effect? Quantitative measurements tend to be much easier to collect and analyze than qualitative measurements. Oftentimes you can let it all run on autopilot, whereas doing good qualitative work always requires concentration, effort, and expertise.
miki123211 5y ago
I believe this effect greatly contributes to why government IT systems are so bad. When you're writing a procurement contract, it's relatively easy to describe what the requested system must do, but almost impossible to enforce a great UI design, as great UI design isn't objectively measurable. As a contractor, you're optimizing for minimum money spent, so if good design is not required, good design gets sacrificed first. One solution to this specific problem would be to conduct user surveys on how pleasant the system is to use, requiring a specific score before the contract is deemed completed. This trend manifests more generally in bigger organizations. Smaller orgs let people judge things subjectively, so all possible aspects are taken into account, making those things relatively good; this is why startups succeed. In a bigger org, there are often objective judgement measures to prevent the influence of personal biases, politics or even bribes. However, those measures poorly reflect how good the thing in question actually is. This is why a big corp might produce worse software, even when competing against a small and underfunded startup. As an example, Apple exempted the first iPhone crew from most internal company procedures, creating a quasi-startup inside Apple. Steve Jobs always had the final say, and his opinions were based on what he thought personally, not on how many points in a requirements specification were satisfied. I believe this was one of the reasons for the iPhone's success.
- cratermoon 5y ago
  > government IT systems This is not limited to governments. Although it's a common naive bias to assert that governments are worse and less efficient than private industry, what is really happening is that government budgets and projects are open to the public, done in the open. For every failed Healthcare.gov, there are dozens of private industry failures that don't make the news because the operations are not subject to the public disclosure rules.
cratermoon 5y ago
Also https://en.wikipedia.org/wiki/Campbell%27s_law https://en.wikipedia.org/wiki/Campbell%27s_law
paulddraper 5y ago
I recall a presentation by a feature team: "And making the share button bright blue yield a 15% increase in share rate." Okay, yeah I'm sure it does. Because it looks terrible and clashes with literally everything on the screen.
snidane 5y ago
I've been looking for the name of this for years. It's everywhere. People focusing only on things they can measure instead of what is important.

ghaff 5y ago

It's a genuinely hard problem because the quantifiable output metrics are easy to measure and an individual often does have some level of direct control over them. So we convince ourselves that they're a reasonable proxy for something we care about but aren't sure how to measure and that we have control systems in place, whether management or individual responsibility, that largely prevent e.g. quality being thrown away in pursuit of quantity. And we're often not entirely wrong if we do pick reasonable proxies and have reasonable control systems in place. Because throwing up our hands and saying metrics are useless is usually not the answer either.

TeMPOraL 5y ago
A flavor of this problem is what could be called "diffusion of responsibility", for a lack of better term. An individual who defined the measure and then optimizes for it will quickly figure out when their measure stops being a good proxy. But in organizations there usually isn't a single person who both understands what the measures are proxying, and has the power to remove a metric once it is used up, or get people to stop overfitting it.
- Jtsummers 5y ago
  Another thing I've observed is when you have two measures that operate at different time scales. Both may even be valid measures, but the one measured (and responded to) more often has a stronger impact, and can negatively impact the less frequently measured metric when there's a conflict between them. A particular instance for this has been (to keep it simple) quantity (speed) and quality in production environments (factories and the like). Daily throughput measures paired with less frequent quality measures. The desire is to keep throughput high, and quality ends up suffering as a result. By integrating quality measures into the process you make the two measures compete on more equal footing, forcing a balance. At least one factory I worked in (well, adjacent to, I was in the software portion not the assembly line) massively reduced their quality problems by integrating quality checks between each station. This contrasted with the prior years where throughput, being measured and reacted to daily, drove them to make things so fast that they had piles of rework at the end. Integrating the quality measures between stations slowed them down, but their rework numbers turned into a rounding error (over a decade ago so I've forgotten the exact numbers, but they went from having items needing rework nearly every day to maybe one or two a month). As a result their real (deliverable to customers) production increased and their cost per unit dropped.
  - AceyMan 5y ago
    I see this as the root cause of the recently announced class action suit against LADWP over their implementation of tiered electricity pricing. The tiers (kwh rates) are in hunks of the a day measured in hours. But the reporting is only available to the consumer in the form of a monthly bill, so by the time you discover you were eating pixies in the Peak Cost hours the heat wave is over and your bill is already through the roof. (Any local SoCal residents please feel free to pick my analysis apart, but that was my first take when I heard about the legal action.)
serverholic 5y ago
The better option is to put someone in charge with a vision. However that is risky.

cryptica 5y ago

To make matters worse, it's also a vicious cycle. For example, if everyone is focused only on specific kinds of data and ignores observable reality, the trend in the data will become self-fulfilling until a point when the perceived inconsistency between the data and observable reality has grown so large that it becomes impossible to ignore. To understand what the big problems are today, you just have to think about the kinds of data which people in government (and the public) haven't been thinking about or aiming for. For example: Happiness, honesty, altruism, sanity... These are not measured and not targeted so they got completely crushed. In the past, large, powerful religious groups would target these characteristics but nowadays, society is more secular so these aspects of our lives have suffered significantly.

serverholic 5y ago
That's one of the thing Andrew Yang talks about. GDP isn't the only thing that matters, nor is it the best metric.

serverholic 5y ago

Another problem is that overly focusing on metrics means that people with good instincts are brought down to the same level as those with poor instincts. As long as metrics are increasing someone can write shit code and design a bloated product.

BuyMyBitcoins 5y ago
“Instinct” can’t be quantified and it’s such a shame that people with good track records can’t quite justify their choices quantitatively. This happens where I work constantly. The managers who get most of their projects delivered on time - I suspect because they have good intuition about effort levels - are constantly struggling to justify their higher initial estimates because the data driven people try to reduce development down to a formula.

burnafter182 5y ago

As it's told by Mandelbrot, the January effect was identified. Rather quickly it was exploited to the point where it no longer existed because the market exerted pressures to counteract it in the process of exploiting it. "Consider three cases. First, suppose a clever chart-reader thinks he has spotted a pattern in the old price records—say, every January, stock prices tend to rise. Can he get rich on that information, by buying in December and selling in January? Answer: No. If the market is big and efficient then others will spot the trend, too, or at least spot his trading on it. Soon, as more traders anticipate the January rally, more people are buying in December—and then, to beat the trend for a December rally, in November. Eventually, the whole phenomenon is spread out over so many months that it ceases to be noticeable. The trend has vanished, killed by its very discovery. In fact, in 1976 some economists spotted just such a pattern of regular January rallies in the stocks of small companies. Many investors close their losing positions towards the end of the year so they can book the loss as a tax deduction—and the market rebounds when they reinvest early in the new tax year. The effect is most pronounced on small stocks, which are more sensitive to small money movements. Alas, before you rush out to trade on this trend, you should know that its discovery seems to have killed it. After all the academic hoopla over it, it no longer shows up as clearly in price charts." -Benoit Mandelbrot, The Misbehavior of Markets https://en.wikipedia.org/wiki/January_effect https://en.wikipedia.org/wiki/January_effect

miki123211 5y ago
As far as I'm aware, this is why markets are considered a second-order chaotic system. In those systems, measurements of how the system performs can actually influence what happens next. This is in contrast to first order systems, i.e. the weather, which are hard to simulate, but the results of the simulations don't affect their accuracy.
Stratoscope 5y ago
This reminds me so much of a Christmas present hack my sister and I invented when we were kids. We used to open all our presents on Christmas morning. Then we begged "Can't we open a couple of presents on Christmas Eve?" So we got to open a few that night. Next year was "Well, how about Christmas Eve Morning? Maybe just one or two?" And the next year was "The 23rd is practically Christmas Eve, isn't it? It's just a few hours apart. Can't we open all our presents on the evening of the 23rd?" And we did! We didn't push it past that: we were already so happy that we got our presents a day and a half before all our friends!

rossdavidh 5y ago

In the context of manufacturing, W.E.Deming said something similar: "that which gets measured, gets improved". His conclusion from this was a little different than McNamara's, though. Since you will inevitably want to track your progress, make sure you track as many things as possible, because anything which is not tracked will get sacrificed to that which is. Up to a point, it's true. One issue is that some things, like vulnerability to supply chain disruptions, are intrinsically harder to track because they are based on rare occurrences. Thus, they will tend to get sacrificed in favor of measures which are more frequent, leading to an emphasis on short-term strategies.

CalChris 5y ago
Actually, Deming said much the opposite: It is wrong to suppose that if you can’t measure it, you can’t manage it – a costly myth. p. 26 of The New Economics for Industry, Government, Education It wasn't Drucker either. https://medium.com/centre-for-public-impact/what-gets-measured-gets-managed-its-wrong-and-drucker-never-said-it-fe95886d3df6 https://medium.com/centre-for-public-impact/what-gets-measur... This whole mindless must measure, must measure mentality has been criticized since the 50s. Measurement is a tool. There are many tools.
- rossdavidh 5y ago
  In my experience, if it isn't measured, then it is assumed that the policy (whatever it is) is working. If you don't measure, you can't be surprised by "wow, it didn't work like we thought". Therefore, mistakes don't get recognized or corrected. There are many tools, but measurement is one of the only ones that brings unexpected bad news to the user, and that is invaluable.
- mumblemumble 5y ago
  I would argue that it's worthwhile to measure as much as you can, insofar as it facilitates orderly decisionmaking processes. The problem is that people tend to think that all measurement is necessarily quantitative. I think that this might be a version of the streetlight effect? Quantitative measurements tend to be much easier to collect and analyze than qualitative measurements. Oftentimes you can let it all run on autopilot, whereas doing good qualitative work always requires concentration, effort, and expertise.

miki123211 5y ago

I believe this effect greatly contributes to why government IT systems are so bad. When you're writing a procurement contract, it's relatively easy to describe what the requested system must do, but almost impossible to enforce a great UI design, as great UI design isn't objectively measurable. As a contractor, you're optimizing for minimum money spent, so if good design is not required, good design gets sacrificed first. One solution to this specific problem would be to conduct user surveys on how pleasant the system is to use, requiring a specific score before the contract is deemed completed. This trend manifests more generally in bigger organizations. Smaller orgs let people judge things subjectively, so all possible aspects are taken into account, making those things relatively good; this is why startups succeed. In a bigger org, there are often objective judgement measures to prevent the influence of personal biases, politics or even bribes. However, those measures poorly reflect how good the thing in question actually is. This is why a big corp might produce worse software, even when competing against a small and underfunded startup. As an example, Apple exempted the first iPhone crew from most internal company procedures, creating a quasi-startup inside Apple. Steve Jobs always had the final say, and his opinions were based on what he thought personally, not on how many points in a requirements specification were satisfied. I believe this was one of the reasons for the iPhone's success.

cratermoon 5y ago
> government IT systems This is not limited to governments. Although it's a common naive bias to assert that governments are worse and less efficient than private industry, what is really happening is that government budgets and projects are open to the public, done in the open. For every failed Healthcare.gov, there are dozens of private industry failures that don't make the news because the operations are not subject to the public disclosure rules.

cratermoon 5y ago

Also https://en.wikipedia.org/wiki/Campbell%27s_law https://en.wikipedia.org/wiki/Campbell%27s_law

paulddraper 5y ago

I recall a presentation by a feature team: "And making the share button bright blue yield a 15% increase in share rate." Okay, yeah I'm sure it does. Because it looks terrible and clashes with literally everything on the screen.

snidane 5y ago

I've been looking for the name of this for years. It's everywhere. People focusing only on things they can measure instead of what is important.