3 ms·

" Managers/bureaucrats want to make decisions based on 'metrics', primarily because it gives them cover if it happens to be the wrong decision" My word, no, th

" Managers/bureaucrats want to make decisions based on 'metrics', primarily because it gives them cover if it happens to be the wrong decision" My word, no, they want to make decisions based on 'metrics' because it validates the 'do or not decision'. We make hardware based products and have a very real need to model component prices, fluctuations, trade/political risks etc.. Most things are not 'art'. If someone wants metrics on something so fuzzy, put the 'wide error' bars on the estimation. Better yet - don't even talk about measurements, make it a 'risk assessment' exercise and identify the key issues that go into defining the unknown. Ontario government is spending a huge amount of money doing 'environmental assessment' for high speed rail - it's better to do that than to start building and hope they don't come across some 'magical wetland' etc. Granted - they are spending too much money on the assessment ...

Eddy_Viscosity2 4y ago
I am in no way advocating for total `gut-based' decision making, because that would be an obvious disaster. I think gathering data, analyzing that data, and making a decision based on the results is a very good and important thing. My issue is that not ALL issues can be decided in this way. So while most things are not 'art', as you've said, some very much are like art and its for these types of issues the metrics model breaks down. It's just the wrong tool for the job. The `risk assessment' approach, I agree can also be a useful one in many cases but not all. I think one example is when trying to measure, or more likely, rank, people for suitability for something. This can be more of an art. There are metrics that can be useful, but then Goodhart's Law [1] can kick in. And even when the metrics are useful, they simply can't account for, and will never be able to account for, the shear complexity and adaptability of actual people. So yes, metrics can be good for some thing, risk assessment good for others, but that still leaves some problems on the table for which neither are sufficient. [1] https://www.formpl.us/blog/goodharts-law#:~:text=Goodhart's%20Law%20is%20the%20idea,they're%20trying%20to%20improve https://www.formpl.us/blog/goodharts-law#:~:text=Goodhart's%....

Eddy_Viscosity2 4y ago

I am in no way advocating for total `gut-based' decision making, because that would be an obvious disaster. I think gathering data, analyzing that data, and making a decision based on the results is a very good and important thing. My issue is that not ALL issues can be decided in this way. So while most things are not 'art', as you've said, some very much are like art and its for these types of issues the metrics model breaks down. It's just the wrong tool for the job. The `risk assessment' approach, I agree can also be a useful one in many cases but not all. I think one example is when trying to measure, or more likely, rank, people for suitability for something. This can be more of an art. There are metrics that can be useful, but then Goodhart's Law [1] can kick in. And even when the metrics are useful, they simply can't account for, and will never be able to account for, the shear complexity and adaptability of actual people. So yes, metrics can be good for some thing, risk assessment good for others, but that still leaves some problems on the table for which neither are sufficient. [1] https://www.formpl.us/blog/goodharts-law#:~:text=Goodhart's%20Law%20is%20the%20idea,they're%20trying%20to%20improve https://www.formpl.us/blog/goodharts-law#:~:text=Goodhart's%....