3 ms·
That may or may not be true depending on how the program is written. All of the math can be correct, but make the mistake of relying on floating point math havi
by MertsA 9y ago
That may or may not be true depending on how the program is written. All of the math can be correct, but make the mistake of relying on floating point math having the associative and distributive properties (it doesn't). I'd imagine the most trouble prone part of the fuel load calculation to be the cancellation error introduced by naively determining the derivative of some function.
E.g.
step = 0.001
fprime = (f(x+step) - f(x)) / step
Having a tiny step value means substantially worse error due to rounding. Instead of being a tiny fraction of a percent off now you've got something potentially dangerous like 30% off. You can't just handwave away loss of precision as something to ignore, there are a handful of common pitfalls that will amplify the total error to the point where it's unacceptably high and not intuitive to figure out the cause of the error.
- _0ffh 9y agoYeah, it' simple to see in this example. Though it's often harder to see in practice, I once heard/read some interesting advice: Test your software in all four rounding modes. If the software works correctly (and approximately agrees) in all four modes, you're good. (I suppose this assumes that you've got reasonable test coverage.)