4 ms·

I think it's worse than this, at least in some fields. In my field, aerospace engineering, the undergraduate curriculum typically has a numerical methods course

I think it's worse than this, at least in some fields. In my field, aerospace engineering, the undergraduate curriculum typically has a numerical methods course. But the course is usually focused on basic algorithms — Newton's method, basic numerical integrqtion schemes, etc. — and does not go deeply enough to allow one to derive or understand the numerical methods used in the design of actual aircraft or spacecraft (FEM, solutions to the Navier Stokes equations, solutions to the PDEs that describe thermal transfer, etc.) so the software that performs these calculations is often treated by engineers as a black box. Admittedly, in a well run project there is usually hardware testing to back up the analysis, but such testing cannot realistically reproduce the conditions a spacecraft will encounter on orbit, or that an aircraft will encounter in near-crash situations. I once sat on the external departmental review board for my alma mater, along with Mike Griffin (former administrator of NASA). We were interviewing recent graduates, and I brought this point up and got the interviewee to admit that he did not understand and could not double check the results of his FEM analysis tool. I was trying to get the board to rcommend beefing up the numerical methods portion of the curricula. Dr. Griffin, on hearing the interviewee admit this, agreed to add the recommendation. But nothing changed in the curriculum.

btrettel 8y ago
If I were every to have input into the curriculum for an engineering program, I'd stress adding more about verification and validation of models, not adding more on numerical methods in general. All users of off-the-shelf models must do at least some sort of check on the model. You do not need to understand how the model works to check it, fortunately. Unfortunately, it's rare in my experience for engineers to do V&V. It might take some culture changes to fix this.
- GlenTheMachine 8y ago
  Yeah, that's a great point. In the example I had in mind, finite elements, the actual model is typically autogenerated by the software package. The CAD model is not meshed by hand. This issue was alluded to in the OP as well. But you are correct that this issue is not really with understanding the numerical method, it's with the model setup.
- wenc 8y ago
  Agree with this. Adding more numerical methods at the undergraduate level may or may not achieve the desired result due to limited time in undergrad and lack of mathematical maturity among undergrads. It might end up being just another thing to cram. I took a number of numerical analysis courses as a graduate student, and we had an entire course on how stuff could go wrong in numerics, e.g. catastrophic cancellation, ill-conditioning, nonconvexity, approximation errors due to discretization, etc. I don't think I could have appreciated all of this as an undergrad because I didn't see the practicality of this stuff until I had to solve large-scale numerical optimization problems as a grad student. Then it all started to come together. Also there was more time in grad school to step back and think more deeply about problems, as opposed to undergrad where you're just rushing to get as many courses done as possible. Furthermore, numerical computation is as much an art as a science. The only way to learn is through the repeated grind of trying and failing, and there's no way to fit that kind of practice into an undergrad curriculum. Most people learn this stuff on the job or in grad school. The role of an undergrad numerical methods course is usually only as an introduction to the landscape. In engineering, V&V requires an intuitive understanding of the physical system at hand, which requires experience, which in turn is usually acquired on the job. This is another reason why it is not taught in undergrad. These types of intuitions are best acquired in industry.
lordnacho 8y ago
> the software that performs these calculations is often treated by engineers as a black box IMO this is a deficiency in how programming is treated in engineering courses. It's there, but people are not taught to apply it to everything else, in the way they are with the math courses. In my degree we did finite element methods, PDEs, and a load of other things, but only in a presentation style. So you'd go into some depth about how the equations worked, but the examples were always little atoms of a real model. So there'd be some diagram of how one element meshes with another and you could do some hand calculations to see what happens. I think the issue is that you have to be a pretty good programmer to understand both the new FE/PDE/etc material you've been shown as well as be able to abstract it to such a level that you can write sensible code for it. If you're not quite proficient with abstraction, a specific language, as well as the coding toolchain, everything takes a long time and you'll be thinking about how the debugger works rather than how the model works.
deleted 8y ago
[deleted]
jacquesm 8y ago
If you think that is bad wait until you see biologists use software. They do not treat it as a black box, they treat it as a god. Obviously this will not hold for all biologists but the things I have seen horrify me that that is part of science. Output from complex pipelines of software is taken as gospel without any checking because ´who can check all of that´.
- nonbel 8y ago
  I've seen bio data get messed up just by one person opening a csv in excel: https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-1044-7 https://genomebiology.biomedcentral.com/articles/10.1186/s13... I mean not even editing anything, just opened the file to look at it and then allowed saving the file on close. But yea, your typcial bio analysis is a lot of clicking around in one peice of software, importing that output into the next software, etc and is totally unreproducible. Then in the end you get an average and standard deviation (at least its not usually only p-values anymore) but no one ever checked what the underlying data looked like. "Is your standard deviation for Group X so large because of low outliers, high outliers or is it a multimodal distribution?" "No idea. Whats a distribution?"
noobermin 8y ago
A lot of code is treated like a black box. For example, can you describe me how ssh works other than schematically? Do you know the details of the cryptographic bits? What you asked for in terms of the fundamental numerical algorithms like FEM is somewhat close to this. They hopefully (especially if they are trying to get a graduate degree) should be able to schematically describe it, but in no way should every scientist have to reinvent FEM or finite difference methods again and again just like people shouldn't implement their own crypto: it can be rather hairy and making a mistake could be disastrous. Writing complicated algorithms should be left to experts who can be trusted.
- a1369209993 8y ago
  > Writing complicated algorithms should be left to experts who can be trusted. How exactly do you propose we get those experts, if colleges aren't capable of training them?
  - esmi 8y ago
    This may have been the case when this article was written but I don’t think it’s the case now. Many universities have graduate and undergraduate computational science programs now. https://www.siam.org/Students-Education/Resources/For-Undergraduate-Students/Detail/graduate-and-undergraduate-programs-in-computational-science https://www.siam.org/Students-Education/Resources/For-Underg...
  - wenc 8y ago
    > How exactly do you propose we get those experts, if colleges aren't capable of training them? In the majority of cases, colleges (undergrad) aren't in fact capable of training them. The material is too specialized for undergrad. However colleges can prepare those who are talented and interested in the subject matter for further studies. Many experts in numerical analysis start specializing in graduate school, and then acquire their expertise through postdocs, academia and industry experience.
- bamboozled 8y ago
  Nooberim, I think many people could actually explain (in gross detail) how SSH works by the way.
8note 8y ago
my mechanical engineering curriculum included a basic numerical methods course in 3rd year along with a finite difference heavy heat transfer course, then optional courses on FEA and CFD in fourth year, that get you through the theoretical backing and writing your own solvers. I'd assume any accredited Canadian university would have the same

btrettel 8y ago

If I were every to have input into the curriculum for an engineering program, I'd stress adding more about verification and validation of models, not adding more on numerical methods in general. All users of off-the-shelf models must do at least some sort of check on the model. You do not need to understand how the model works to check it, fortunately. Unfortunately, it's rare in my experience for engineers to do V&V. It might take some culture changes to fix this.

GlenTheMachine 8y ago
Yeah, that's a great point. In the example I had in mind, finite elements, the actual model is typically autogenerated by the software package. The CAD model is not meshed by hand. This issue was alluded to in the OP as well. But you are correct that this issue is not really with understanding the numerical method, it's with the model setup.
wenc 8y ago
Agree with this. Adding more numerical methods at the undergraduate level may or may not achieve the desired result due to limited time in undergrad and lack of mathematical maturity among undergrads. It might end up being just another thing to cram. I took a number of numerical analysis courses as a graduate student, and we had an entire course on how stuff could go wrong in numerics, e.g. catastrophic cancellation, ill-conditioning, nonconvexity, approximation errors due to discretization, etc. I don't think I could have appreciated all of this as an undergrad because I didn't see the practicality of this stuff until I had to solve large-scale numerical optimization problems as a grad student. Then it all started to come together. Also there was more time in grad school to step back and think more deeply about problems, as opposed to undergrad where you're just rushing to get as many courses done as possible. Furthermore, numerical computation is as much an art as a science. The only way to learn is through the repeated grind of trying and failing, and there's no way to fit that kind of practice into an undergrad curriculum. Most people learn this stuff on the job or in grad school. The role of an undergrad numerical methods course is usually only as an introduction to the landscape. In engineering, V&V requires an intuitive understanding of the physical system at hand, which requires experience, which in turn is usually acquired on the job. This is another reason why it is not taught in undergrad. These types of intuitions are best acquired in industry.

lordnacho 8y ago

> the software that performs these calculations is often treated by engineers as a black box IMO this is a deficiency in how programming is treated in engineering courses. It's there, but people are not taught to apply it to everything else, in the way they are with the math courses. In my degree we did finite element methods, PDEs, and a load of other things, but only in a presentation style. So you'd go into some depth about how the equations worked, but the examples were always little atoms of a real model. So there'd be some diagram of how one element meshes with another and you could do some hand calculations to see what happens. I think the issue is that you have to be a pretty good programmer to understand both the new FE/PDE/etc material you've been shown as well as be able to abstract it to such a level that you can write sensible code for it. If you're not quite proficient with abstraction, a specific language, as well as the coding toolchain, everything takes a long time and you'll be thinking about how the debugger works rather than how the model works.

deleted 8y ago

[deleted]

jacquesm 8y ago

If you think that is bad wait until you see biologists use software. They do not treat it as a black box, they treat it as a god. Obviously this will not hold for all biologists but the things I have seen horrify me that that is part of science. Output from complex pipelines of software is taken as gospel without any checking because ´who can check all of that´.

nonbel 8y ago
I've seen bio data get messed up just by one person opening a csv in excel: https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-1044-7 https://genomebiology.biomedcentral.com/articles/10.1186/s13... I mean not even editing anything, just opened the file to look at it and then allowed saving the file on close. But yea, your typcial bio analysis is a lot of clicking around in one peice of software, importing that output into the next software, etc and is totally unreproducible. Then in the end you get an average and standard deviation (at least its not usually only p-values anymore) but no one ever checked what the underlying data looked like. "Is your standard deviation for Group X so large because of low outliers, high outliers or is it a multimodal distribution?" "No idea. Whats a distribution?"

noobermin 8y ago

A lot of code is treated like a black box. For example, can you describe me how ssh works other than schematically? Do you know the details of the cryptographic bits? What you asked for in terms of the fundamental numerical algorithms like FEM is somewhat close to this. They hopefully (especially if they are trying to get a graduate degree) should be able to schematically describe it, but in no way should every scientist have to reinvent FEM or finite difference methods again and again just like people shouldn't implement their own crypto: it can be rather hairy and making a mistake could be disastrous. Writing complicated algorithms should be left to experts who can be trusted.

a1369209993 8y ago
> Writing complicated algorithms should be left to experts who can be trusted. How exactly do you propose we get those experts, if colleges aren't capable of training them?
- esmi 8y ago
  This may have been the case when this article was written but I don’t think it’s the case now. Many universities have graduate and undergraduate computational science programs now. https://www.siam.org/Students-Education/Resources/For-Undergraduate-Students/Detail/graduate-and-undergraduate-programs-in-computational-science https://www.siam.org/Students-Education/Resources/For-Underg...
- wenc 8y ago
  > How exactly do you propose we get those experts, if colleges aren't capable of training them? In the majority of cases, colleges (undergrad) aren't in fact capable of training them. The material is too specialized for undergrad. However colleges can prepare those who are talented and interested in the subject matter for further studies. Many experts in numerical analysis start specializing in graduate school, and then acquire their expertise through postdocs, academia and industry experience.
bamboozled 8y ago
Nooberim, I think many people could actually explain (in gross detail) how SSH works by the way.

8note 8y ago

my mechanical engineering curriculum included a basic numerical methods course in 3rd year along with a finite difference heavy heat transfer course, then optional courses on FEA and CFD in fourth year, that get you through the theoretical backing and writing your own solvers. I'd assume any accredited Canadian university would have the same