This is an excellent example of Goodhart's Law in practice. Every time it comes up, I'm reminded of a Dune quote: "The problem isn't thinking machines, it's le

This is an excellent example of Goodhart's Law in practice. Every time it comes up, I'm reminded of a Dune quote: "The problem isn't thinking machines, it's letting the machines do the thinking for you." IMO the Law comes into effect as people stop also paying attention to the externalities in favor of "letting the [metric] do the thinking for them.".