Calling this the "strong version of Goodhart's law" was immediately brain-expanding for me. I have been thinking of goodhart's law a lot, but realized I had be

Calling this the "strong version of Goodhart's law" was immediately brain-expanding for me. I have been thinking of goodhart's law a lot, but realized I had been leaning toward focusing on human reaction to the metric as the cause of it; but this reminded me it's actually fundamentally about the fact that any metric itself is inherently not an exact representation of the quality you wish to represent. And that this may, as OP argues, make goodhart's law fundamental to pretty much any metric used as a target. Independently of how well-intentioned any actors. It's not a result of like human laziness or greed or competing interests, it's an epistemic (?) result of the neccesary imperfection of metric validity. This makes some of OP's more contentious "Inject noise into the system" and "early stopping" ideas more interesting even for social targets. "The more our social systems break due to the strong version of Goodhart's law, the less we will be able to take the concerted rational action required to fix them." Well, that's terrifying.