4 ms·
That's a classic example of Goodhart's law. Regulations call for extremely specific tests to measure crashworthiness, so vehicles are engineered to pass those t
by hammock 10y ago
That's a classic example of Goodhart's law. Regulations call for extremely specific tests to measure crashworthiness, so vehicles are engineered to pass those tests (front-end impact with a specific wall shape at a specific velocity and angle, etc), so they get a "five star crash rating" but at the expense of real-world crash performance.
- Xylakant 10y agoWhat else could regulations call for than a defined set of tests? You need comparability between tests or the tests are worthless when it comes to informing customers. Real world crashes differ wildly, so it's impossible to use those as a measure. There are also many different crash tests carried out around the globe and a car performing well in many of those will perform well in the real world.
- csours 10y agoUse a defined set of tests along with real world metrics. [1] Ensure your tests match reality as close as possible. Ensure you test for the most important events found in the real world. In the case of crash tests, make sure you test for crashes that often cause death or serious injury. Real world metrics get really tricky and expensive, but when speaking about life and safety, it's probably important enough to measure. It's also very important to be smart about measurement and reporting of events, statistics, and trends. 1. http://lesswrong.com/lw/1ws/the_importance_of_goodharts_law/ http://lesswrong.com/lw/1ws/the_importance_of_goodharts_law/
- hammock 10y agoGoodwin's law outlines a paradox. Practically, there is no perfect metric; every metric has a limited shelf life.
- dpark 10y ago> at the expense of real-world crash performance. Citation? I'm pretty sure that the crash ratings system has resulted in better real-world crash performance.
- Xylakant 10y agoIt's theoretically possible that crash-tests overshoot the optimum and thus effort that could be used for better real world performance gets wasted.
- dpark 10y agoIt's possible. I'm not sure it's likely. Even if it is true, we're still better off having cars optimized for crash survival even if the optimization parameters are imperfect.
- carterehsmith 10y ago> so they get a "five star crash rating" but at the expense of real-world crash performance Do you have any source for that?