I'm going to go on a rant here. Most times when I hear someone invoke Goodhart's law, it's because they are opposed to transparent and metric-driven decision-ma

I'm going to go on a rant here. Most times when I hear someone invoke Goodhart's law, it's because they are opposed to transparent and metric-driven decision-making. It's easy to find the flaws with this approach, as the article has effectively done. But what they inevitably ignore is the flaws of not using this approach. Without a transparent metrics-driven decision making process, what you end up with one of the following: - A totally opaque use of metrics to make decisions. You as a customer-support rep are trying to answer as many customer tickets as possible with a reasonably high satisfaction rate, because you think that's what you're supposed to do. Except that management has decided that you will secretly be judged based solely on satisfaction rates, and not the volume of work done. So you keep getting passed over for promotions and raises, losing out to others who are closing far fewer tickets but with a marginally higher satisfaction rate. And you have no idea why - A totally subjective decision-making process filled with all manner of cognitive biases. You have an accent, so you get passed over for promotions. Your buddy comes to work dressed very sharp, so he gets great performance reviews. The new guy goes bowling with your boss every Sunday, and gets fast tracked for promotions. Nobody knows or says these reasons. Everyone can point after-the-fact to some piece of data that explains why the guy with an accent would not be a good leader, and why the manager's best friend deserves a promotion. And yet, you keep noticing consistently that some people get ahead far more than their talents would suggest - No decision making process at all. The guy next to you who spends half the work day hungover? Still around 10 years later. The guy who customers hate talking to because he's barely helpful? Clocks in 9-5 everyday with no problems at all. Both of them have abysmal work output or customer satisfaction ratings? Well, you've heard of Goodhart's law right. We can't just go around evaluating people based on these very flawed and could-be-gamed metrics Yes, it's extremely hard finding a good measure to optimize for. And yes, there are second order effects where people will try to game any metrics you start measuring. But how does this compare against the problems involved in NOT doing this? To use the article's parlance, overfitting is bad, but underfitting is also bad. Instead of trotting out Goodhart's law every time someone suggests transparently using metrics to guide decision-making, perhaps we should be discussing how to find the right balance between underfitting and overfitting. And what combination of metrics will produce the best proxy for the goals we're trying to achieve, while avoiding the problems of overfitting to a single metric.