4 ms·

> I immediately glommed onto this list as a more useful formulation than Goodhart’s Law. Joiner’s list suggests a number of solutions: > Make it difficult to di

by tqi 2y ago

> I immediately glommed onto this list as a more useful formulation than Goodhart’s Law. Joiner’s list suggests a number of solutions: > Make it difficult to distort the system. > Make it difficult to distort the data, and If companies knew how to make it difficult to distort the system/data, don't you think they would have done it already? This feels like telling a person learning a new language that they should try to sound more fluent.

llamaimperative 2y ago
If they knew how to do it and that they should. I think Goodhart’s Law is useful to know about because what it’s really suggesting is that people are shockingly good, probably much better than you thought, at distorting the system.
abetusk 2y ago
The article goes into (what I consider) actionable methods. Specifically: * Create a finance department that's independent in both their reporting and ability to confirm metrics reported by other departments * Provide a periodic meeting (for executives/mangers) that reviews all metrics and allows them to alter them if need be * Don't try to provide a small number of measurable metrics or a "north star" single metric The idea being that the review meeting of 500+ gives a better potential model. Further, even though 500+ metrics is a lot to review, each should be reviewed briefly, with most of them being "no change, move on" but allows managers to get a holistic feel for the model and identify metrics that are or are becoming outliers (positively or negatively correlated). The independent finance department means that the reporting of bad data is discouraged and the independent finance department coupled with the WBR and its subsequent empowerment, allow for facilities to change the system. The three main points (make difficult to distort the system, distort the data and provide facilities for change) need to be all implemented to have an effect. If only the "punishment" is provided (making it difficult to distort the system/data) without any facility for change is putting too much pressure without any relief.

by tqi 2y ago

llamaimperative 2y ago

If they knew how to do it and that they should. I think Goodhart’s Law is useful to know about because what it’s really suggesting is that people are shockingly good, probably much better than you thought, at distorting the system.

abetusk 2y ago

The article goes into (what I consider) actionable methods. Specifically: * Create a finance department that's independent in both their reporting and ability to confirm metrics reported by other departments * Provide a periodic meeting (for executives/mangers) that reviews all metrics and allows them to alter them if need be * Don't try to provide a small number of measurable metrics or a "north star" single metric The idea being that the review meeting of 500+ gives a better potential model. Further, even though 500+ metrics is a lot to review, each should be reviewed briefly, with most of them being "no change, move on" but allows managers to get a holistic feel for the model and identify metrics that are or are becoming outliers (positively or negatively correlated). The independent finance department means that the reporting of bad data is discouraged and the independent finance department coupled with the WBR and its subsequent empowerment, allow for facilities to change the system. The three main points (make difficult to distort the system, distort the data and provide facilities for change) need to be all implemented to have an effect. If only the "punishment" is provided (making it difficult to distort the system/data) without any facility for change is putting too much pressure without any relief.