4 ms·

Any good tricks for catching these sorts of errors? I'm a theoretical fluid dynamicist. When doing math I'll typically check the dimensions, order of magnitude

Any good tricks for catching these sorts of errors? I'm a theoretical fluid dynamicist. When doing math I'll typically check the dimensions, order of magnitude, bounds, functional dependencies (anything missing or something there that shouldn't be?), overall trends of an equation, and limiting cases. For solutions to equations it's useful to also plug the apparent solution back into the equation. Another check ideally comes after waiting a while so that my mind is fresh. Sometimes what I did seems dumb by then. Coding up the math is useful too. I've found that I can never add too many assertions. I might have caught the error described in the article as I often use assertions to check bounds. Peer review isn't as reliable as I'd like. When it comes to math, most people seem to "rubber stamp" it. I can recall reviewing a paper that had math problems that were fairly obvious to me. For some reason I got the reports from the other reviewers, and I was the only one to notice...

cm2187 7y ago
Intuitively, only store numbers in their natural dimension. The same thing regularly happens in finance where interest rates (1.5% ie 0.015) is often stored as percent (1.5) in databases. And of course you never know what conventions people take, particularly in these days of super low interest rates. Then you have spreads which can either be stored in basis points (10bps), percent (0.1%) or natural (0.001). And currency of amounts, etc. All table columns should also have unit in names, whether in charts, tables, database or variable names.
- TeMPOraL 7y ago
  > All table columns should also have unit in names, whether in charts, tables, database or variable names. Oh yes. That would mitigate a lot of stupidity. Imagine if math in programming and Excel had to have both a) always visible/known units, b) unit propagation and c) unit checks. So if A is seconds and B is kilograms, the compiler/spreadsheet wouldn't let you write C = A + B.
  - selimthegrim 7y ago
    Blockpad (http://www.blockpad.net http://www.blockpad.net) is the answer to your prayers.
  - jmnicolas 7y ago
    Yet some people argue against strongly typed language ...
- Doctor_Fegg 7y ago
  > All table columns should also have unit in names Very similar to Hungarian notation: https://en.m.wikipedia.org/wiki/Hungarian_notation https://en.m.wikipedia.org/wiki/Hungarian_notation
  - signa11 7y ago
    so then, 'paul' stands for a 'pointer to a unsigned long' :)
cbanek 7y ago
Have someone else independently code up / compute the number from scratch and compare against it. Likely you'll come up with a different number, then figure out which one is right (you'll probably find both are still wrong).
piertoni 7y ago
Those errors are the ones that you catch by experience. That means making a comparison with previous done (and well finished) jobs. a When I refer only at theory I feel unsafe for the final result, so I try to search for similar jobs or asks experienced workers. I am sure that experienced workers also in this case of the submarine were complaining that was too heavy. Experienced workers base their "calculations" work on previous done jobs and this is really good to give an idea about if is going to work.
monster_group 7y ago
Assertions is the way to go. In every domain there are invariants. Check for those invariants after any update to your model. For example - in the submarine case, the two invariants would be - "submarine can float" and "submarine can sink". No matter what your submarine design is, the final product should be able to float and sink as needed and therefore these assertions should always be true. If at any time either of these two invariants are violated, you've got a problem in your design.
mechhacker 7y ago
In my experience reviewing engineering calculations, I end up doing a lot of work either by hand or in something that recognizes scientific/engineering units like MathCad or SMath and that also displays the equations. I have marked up 100+ page reports, checking every number, in red pen and doing the math on a handheld calculator, checking the page references for all referred equations. The best habit I had, from reviewing others' work, is to give my own work the same treatment. This is the only way I got a long engineering report out in a reasonable amount of time: by catching my own mistakes by actually printing it out and going through it with a fine tooth comb. It takes time, but not as long as you would think in many cases. I've seen others just kind of skim a document and read a few words and see the shape of the equations, then sign off on it for non safety parts. Even then, you can easily lose lots of money on simple mistakes. I've also seen inherited spreadsheets that went through several hands, with magical things added, that the past two people using it never checked. Some people never learn to really go through and check.
perl4ever 7y ago
If you're doing some sort of analysis, there is only one meaningful way to check it - have another equally capable person do it without looking at your work and see if they get the same answer. If (when) they don't, they can look into why. Maybe people in your experience don't do that for math or code, but it's a clear bright line standard, either you do it, because the task is important to get right or you don't, because it doesn't really matter. I have seen where too many layers of checklists and verification just make people assume another layer will catch errors and then nobody checks thoroughly.
- btrettel 7y ago
  > Maybe people in your experience don't do that for math or code, but it's a clear bright line standard, either you do it, because the task is important to get right or you don't, because it doesn't really matter. > I have seen where too many layers of checklists and verification just make people assume another layer will catch errors and then nobody checks thoroughly. Yes, my experience is that few people are willing to do this. The important question then is how to find people willing to redo the work independently. I haven't found anyone who seems interested in doing this or able to dedicate time to this.
  - perl4ever 7y ago
    "how to find people willing to redo the work independently" Taking laborious manual procedures and automating them so as to flush out all the details that were overlooked or gotten incorrect is what I do at the moment.

cm2187 7y ago

Intuitively, only store numbers in their natural dimension. The same thing regularly happens in finance where interest rates (1.5% ie 0.015) is often stored as percent (1.5) in databases. And of course you never know what conventions people take, particularly in these days of super low interest rates. Then you have spreads which can either be stored in basis points (10bps), percent (0.1%) or natural (0.001). And currency of amounts, etc. All table columns should also have unit in names, whether in charts, tables, database or variable names.

TeMPOraL 7y ago
> All table columns should also have unit in names, whether in charts, tables, database or variable names. Oh yes. That would mitigate a lot of stupidity. Imagine if math in programming and Excel had to have both a) always visible/known units, b) unit propagation and c) unit checks. So if A is seconds and B is kilograms, the compiler/spreadsheet wouldn't let you write C = A + B.
- selimthegrim 7y ago
  Blockpad (http://www.blockpad.net http://www.blockpad.net) is the answer to your prayers.
- jmnicolas 7y ago
  Yet some people argue against strongly typed language ...
Doctor_Fegg 7y ago
> All table columns should also have unit in names Very similar to Hungarian notation: https://en.m.wikipedia.org/wiki/Hungarian_notation https://en.m.wikipedia.org/wiki/Hungarian_notation
- signa11 7y ago
  so then, 'paul' stands for a 'pointer to a unsigned long' :)

cbanek 7y ago

Have someone else independently code up / compute the number from scratch and compare against it. Likely you'll come up with a different number, then figure out which one is right (you'll probably find both are still wrong).

piertoni 7y ago

Those errors are the ones that you catch by experience. That means making a comparison with previous done (and well finished) jobs. a When I refer only at theory I feel unsafe for the final result, so I try to search for similar jobs or asks experienced workers. I am sure that experienced workers also in this case of the submarine were complaining that was too heavy. Experienced workers base their "calculations" work on previous done jobs and this is really good to give an idea about if is going to work.

monster_group 7y ago

Assertions is the way to go. In every domain there are invariants. Check for those invariants after any update to your model. For example - in the submarine case, the two invariants would be - "submarine can float" and "submarine can sink". No matter what your submarine design is, the final product should be able to float and sink as needed and therefore these assertions should always be true. If at any time either of these two invariants are violated, you've got a problem in your design.

mechhacker 7y ago

In my experience reviewing engineering calculations, I end up doing a lot of work either by hand or in something that recognizes scientific/engineering units like MathCad or SMath and that also displays the equations. I have marked up 100+ page reports, checking every number, in red pen and doing the math on a handheld calculator, checking the page references for all referred equations. The best habit I had, from reviewing others' work, is to give my own work the same treatment. This is the only way I got a long engineering report out in a reasonable amount of time: by catching my own mistakes by actually printing it out and going through it with a fine tooth comb. It takes time, but not as long as you would think in many cases. I've seen others just kind of skim a document and read a few words and see the shape of the equations, then sign off on it for non safety parts. Even then, you can easily lose lots of money on simple mistakes. I've also seen inherited spreadsheets that went through several hands, with magical things added, that the past two people using it never checked. Some people never learn to really go through and check.

perl4ever 7y ago

If you're doing some sort of analysis, there is only one meaningful way to check it - have another equally capable person do it without looking at your work and see if they get the same answer. If (when) they don't, they can look into why. Maybe people in your experience don't do that for math or code, but it's a clear bright line standard, either you do it, because the task is important to get right or you don't, because it doesn't really matter. I have seen where too many layers of checklists and verification just make people assume another layer will catch errors and then nobody checks thoroughly.

btrettel 7y ago
> Maybe people in your experience don't do that for math or code, but it's a clear bright line standard, either you do it, because the task is important to get right or you don't, because it doesn't really matter. > I have seen where too many layers of checklists and verification just make people assume another layer will catch errors and then nobody checks thoroughly. Yes, my experience is that few people are willing to do this. The important question then is how to find people willing to redo the work independently. I haven't found anyone who seems interested in doing this or able to dedicate time to this.
- perl4ever 7y ago
  "how to find people willing to redo the work independently" Taking laborious manual procedures and automating them so as to flush out all the details that were overlooked or gotten incorrect is what I do at the moment.