5 ms·

They where not back checking against real word experiments?

MaxBarraclough 9y ago
I've heard horror stories of supposedly quite serious software modelling programs behaving quite differently when their input problem was rotated a few degrees.
- lostboys67 9y ago
  Ugh - I remember spending a lot of time checking our pump simulator module against real word experiments back when I worked in technical computing
- dagw 9y ago
  I've heard horror stories of supposedly quite serious software modelling programs behaving quite differently when their input problem was rotated a few degrees. Sounds perfectly normal :) The standard advice here at work for one of our modelling packages is: if it crashes try rotating your input model by one degree and try again. Most of the time that will fix it. If it doesn't fix it try one degree in the other direction.
  - MaxBarraclough 9y ago
    Do people publish work based on this thing? Is correctness somehow assured provided it doesn't crash? Or is the whole thing just a joke?
    - dagw 9y ago
      Do people publish work based on this thing? I'm sure lots of people do. We're a commercial company selling consultancy service around this software (technically around some inhouse tools built on top of this software), among many other things. Is correctness somehow assured provided it doesn't crash? On the whole well tested PDE solvers either crash, give answers that are off by several orders of magnitude, or give a correct answer. (If the answer they give is relevant to what you're trying to model is left as an exercise to the reader) We're reasonably sure that, if the calculations converge, the solution provided is a correct numeric approximation (within the error bounds given) of the PDEs we're claiming to solve. We also believe that the PDEs we're using provide a reasonable balance between modeling what we claim we're modelling and our computation running in a reasonable amount of time.
dagw 9y ago
They probably did at some point in the misty past. The problem is that running relevant experiments is often very expensive (otherwise they wouldn't need to do it in software) and they can only test a tiny handful of cases. More often than not your calibrating your model against another model that was calibrated against another model that was calibrated against a set of simple experiments done in the 60's.

I've heard horror stories of supposedly quite serious software modelling programs behaving quite differently when their input problem was rotated a few degrees.

lostboys67 9y ago
Ugh - I remember spending a lot of time checking our pump simulator module against real word experiments back when I worked in technical computing
dagw 9y ago
I've heard horror stories of supposedly quite serious software modelling programs behaving quite differently when their input problem was rotated a few degrees. Sounds perfectly normal :) The standard advice here at work for one of our modelling packages is: if it crashes try rotating your input model by one degree and try again. Most of the time that will fix it. If it doesn't fix it try one degree in the other direction.
- MaxBarraclough 9y ago
  Do people publish work based on this thing? Is correctness somehow assured provided it doesn't crash? Or is the whole thing just a joke?
  - dagw 9y ago
    Do people publish work based on this thing? I'm sure lots of people do. We're a commercial company selling consultancy service around this software (technically around some inhouse tools built on top of this software), among many other things. Is correctness somehow assured provided it doesn't crash? On the whole well tested PDE solvers either crash, give answers that are off by several orders of magnitude, or give a correct answer. (If the answer they give is relevant to what you're trying to model is left as an exercise to the reader) We're reasonably sure that, if the calculations converge, the solution provided is a correct numeric approximation (within the error bounds given) of the PDEs we're claiming to solve. We also believe that the PDEs we're using provide a reasonable balance between modeling what we claim we're modelling and our computation running in a reasonable amount of time.

dagw 9y ago

They probably did at some point in the misty past. The problem is that running relevant experiments is often very expensive (otherwise they wouldn't need to do it in software) and they can only test a tiny handful of cases. More often than not your calibrating your model against another model that was calibrated against another model that was calibrated against a set of simple experiments done in the 60's.