4 ms·

I don't have a source, but someone said that we don't crash cars for crashworthiness, we crash cars to validate the computer models we have for crashworthiness.

by csours 10y ago

I don't have a source, but someone said that we don't crash cars for crashworthiness, we crash cars to validate the computer models we have for crashworthiness. I don't completely agree with that statement, but I do think it's interesting.

Retric 10y ago
It's more than just validating the computer models, it's also the manufacturing process, and the car company etc. In the end when a third party is crashing a physical object that's much harder to fake.
- csours 10y ago
  Exactly, that's why I don't completely agree with the statement. If you model with high strength steel and the manufacturer or supplier make a mistake or substitution, then your model doesn't represent reality. The crash tests validate reality and the model at the same time.
hammock 10y ago
That's a classic example of Goodhart's law. Regulations call for extremely specific tests to measure crashworthiness, so vehicles are engineered to pass those tests (front-end impact with a specific wall shape at a specific velocity and angle, etc), so they get a "five star crash rating" but at the expense of real-world crash performance.
- Xylakant 10y ago
  What else could regulations call for than a defined set of tests? You need comparability between tests or the tests are worthless when it comes to informing customers. Real world crashes differ wildly, so it's impossible to use those as a measure. There are also many different crash tests carried out around the globe and a car performing well in many of those will perform well in the real world.
  - csours 10y ago
    Use a defined set of tests along with real world metrics. [1] Ensure your tests match reality as close as possible. Ensure you test for the most important events found in the real world. In the case of crash tests, make sure you test for crashes that often cause death or serious injury. Real world metrics get really tricky and expensive, but when speaking about life and safety, it's probably important enough to measure. It's also very important to be smart about measurement and reporting of events, statistics, and trends. 1. http://lesswrong.com/lw/1ws/the_importance_of_goodharts_law/ http://lesswrong.com/lw/1ws/the_importance_of_goodharts_law/
    - Xylakant 10y ago
      But that's exactly what happens: crash tests get validated against real world accidents and refined all the time.
      - csours 10y ago
        Is that a problem? That sounds like a solution... but it's a continuous solution, not a solution that you apply once and forget about.
  - hammock 10y ago
    Goodwin's law outlines a paradox. Practically, there is no perfect metric; every metric has a limited shelf life.
- dpark 10y ago
  > at the expense of real-world crash performance. Citation? I'm pretty sure that the crash ratings system has resulted in better real-world crash performance.
  - Xylakant 10y ago
    It's theoretically possible that crash-tests overshoot the optimum and thus effort that could be used for better real world performance gets wasted.
    - dpark 10y ago
      It's possible. I'm not sure it's likely. Even if it is true, we're still better off having cars optimized for crash survival even if the optimization parameters are imperfect.
- carterehsmith 10y ago
  > so they get a "five star crash rating" but at the expense of real-world crash performance Do you have any source for that?
rdtsc 10y ago
I worked a company what made a popular CAE (Computer Aided Engineering) package. It was really fun to see the connection between physics, material science, mechanical engineering, and software development all in one place. As an intern I remember being fascinated by the ability to build a part, generate a mesh, assign a material to it, assign forced to some parts of it, "solve it" and then watch it deform. It was also fun to use Irix, AIX, HP UX, and SunOS, but the switch to Windows had already started. It is funny to think that at some point, Unix workstation were the top graphics workstations available...

by csours 10y ago

Retric 10y ago

It's more than just validating the computer models, it's also the manufacturing process, and the car company etc. In the end when a third party is crashing a physical object that's much harder to fake.

csours 10y ago
Exactly, that's why I don't completely agree with the statement. If you model with high strength steel and the manufacturer or supplier make a mistake or substitution, then your model doesn't represent reality. The crash tests validate reality and the model at the same time.

hammock 10y ago

That's a classic example of Goodhart's law. Regulations call for extremely specific tests to measure crashworthiness, so vehicles are engineered to pass those tests (front-end impact with a specific wall shape at a specific velocity and angle, etc), so they get a "five star crash rating" but at the expense of real-world crash performance.

Xylakant 10y ago
What else could regulations call for than a defined set of tests? You need comparability between tests or the tests are worthless when it comes to informing customers. Real world crashes differ wildly, so it's impossible to use those as a measure. There are also many different crash tests carried out around the globe and a car performing well in many of those will perform well in the real world.
- csours 10y ago
  Use a defined set of tests along with real world metrics. [1] Ensure your tests match reality as close as possible. Ensure you test for the most important events found in the real world. In the case of crash tests, make sure you test for crashes that often cause death or serious injury. Real world metrics get really tricky and expensive, but when speaking about life and safety, it's probably important enough to measure. It's also very important to be smart about measurement and reporting of events, statistics, and trends. 1. http://lesswrong.com/lw/1ws/the_importance_of_goodharts_law/ http://lesswrong.com/lw/1ws/the_importance_of_goodharts_law/
  - Xylakant 10y ago
    But that's exactly what happens: crash tests get validated against real world accidents and refined all the time.
    - csours 10y ago
      Is that a problem? That sounds like a solution... but it's a continuous solution, not a solution that you apply once and forget about.
- hammock 10y ago
  Goodwin's law outlines a paradox. Practically, there is no perfect metric; every metric has a limited shelf life.
dpark 10y ago
> at the expense of real-world crash performance. Citation? I'm pretty sure that the crash ratings system has resulted in better real-world crash performance.
- Xylakant 10y ago
  It's theoretically possible that crash-tests overshoot the optimum and thus effort that could be used for better real world performance gets wasted.
  - dpark 10y ago
    It's possible. I'm not sure it's likely. Even if it is true, we're still better off having cars optimized for crash survival even if the optimization parameters are imperfect.
carterehsmith 10y ago
> so they get a "five star crash rating" but at the expense of real-world crash performance Do you have any source for that?

rdtsc 10y ago

I worked a company what made a popular CAE (Computer Aided Engineering) package. It was really fun to see the connection between physics, material science, mechanical engineering, and software development all in one place. As an intern I remember being fascinated by the ability to build a part, generate a mesh, assign a material to it, assign forced to some parts of it, "solve it" and then watch it deform. It was also fun to use Irix, AIX, HP UX, and SunOS, but the switch to Windows had already started. It is funny to think that at some point, Unix workstation were the top graphics workstations available...