5 ms·

From what I see helping some friends not well versed with programming doing their PhDs, FORTRAN still has a lot of penetration with scientists. Lots of climate

by piva00 3y ago

From what I see helping some friends not well versed with programming doing their PhDs, FORTRAN still has a lot of penetration with scientists. Lots of climate models are FORTRAN code. Also, through helping them I can see the dire state of scientific code, it's a mess, if departments had the budget to employ 1-2 professional programmers to help them mentor staff it could be extremely helpful for science code to be more easily shared and reasoned about, some of the code I've seen is basically throw away code after the contributors aren't around anymore...

konradha 3y ago
There is project bringing Fortran to the browser, though (_using WebAssembly_) https://dev.lfortran.org/ https://dev.lfortran.org/
mike_hearn 3y ago
A lot of scientific code is worse than a mess, it outright doesn't work and yields incorrect results, a problem which is then routinely covered up. I've seen this first hand :( The worst part is that universities don't realize how much they don't know. On the rare occasions outsiders notice what's going on they are faced with a wall of baffling excuses and justifications for why so much doesn't work, like "if it didn't crash it must be correct" or "scientists don't need unit tests, we just look at the results and know they are right because we're experts". Academics are happy to pronounce that professional coders can't judge the correctness of their work and do so loudly and publicly. As for money, well departments do have the budgets to hire developers. Science funding is in the high billions in most western countries. The problem is not departmental budgets, it's a social problem. Universities love the practice of spreading money amongst as many professors and tiny departments as possible in order to lay claim to every possible area of human knowledge/experience. Combine that with a culture of low standards and coverups and you've got a recipe for disasters (e.g. invariably critical bug fixes that corrupt data are described as not affecting the final results even when that clearly can't be true).
- abyssin 3y ago
  I have the same experience in a psychology faculty. I helped a PhD student who was working on some MATLAB code. The amount of code I had to write for something similar as they were doing was orders of magnitude lower, and faster to write. But they spent months hurting their brains and therefor thought they were doing something really smart. There was a lot of micromanagement of clueless researchers, and the consequence is that everything had to be done the exact same way as the more experienced members of the lab did previously. This was the most wasteful work organization I've seen in my life, but there was little awareness of it, compared to other state-subsidized projects.
- abought 3y ago
  Some departments are starting to hire programmers. There's an effort to define this as a (broad) job category under the heading "Research Software Engineer": https://society-rse.org/ https://society-rse.org/ https://us-rse.org/ https://us-rse.org/ Institutional support varies widely; some projects or teams are rather well funded for big projects and senior talent, while at other schools, the cost structure is more aimed at "one off" projects staffed by more recent graduates. A recent grant is trying to fund this work at several schools with a history of well organized services: https://www.schmidtfutures.org/our-work-old/virtual-institute-for-scientific-software/ https://www.schmidtfutures.org/our-work-old/virtual-institut...
  - mike_hearn 3y ago
    That's been around for a while but the pay for these roles is very uncompetitive, and universities don't care if professors don't do it. It doesn't make any difference to whether you get published or not so it'll probably remain niche. Now, if journals start rejecting papers because the code isn't provided or because it was provided but professional software engineers reviewed it and gave it a thumbs down, that'd change. But journals struggle to keep obviously AI generated material out of their pages, so they aren't going to do anything like that.
    - abought 3y ago
      It depends. Some of the foundation-funded positions are competitive, and a few centers have surprisingly professional leadership. People are trying to organize, and- even if it's an uphill climb- there's been some improvement at the edges. Anecdotally, some of the RSE leads I've spoken to are seeing more long-term demand than they predicted, which might lead to more room for senior roles. Currently quite a few teams (outside the big centers) seem to be priced way too low, usually explained as because they're testing the waters.... so "cheap student labor" and "one off project" is what they can afford. Minor heretical aside: one thing I miss about old twitter is that academia was developing a real "second layer" on top of journals, where things like reproducibility could be discussed publicly. PubPeer is a partial solution, as are GitHub issues... if enough gatekeepy people really see value in code quality, norms will shift with or without mandates.
btrettel 3y ago
I get the impression that you think that Fortran is a bad language for science. Modern Fortran is a nice language for many scientific computing tasks. For me, I like Fortran because it's basically the easiest statically-typed compiled language (I rarely have to think about pointers, for instance), it's basically as fast as C, it has some good features for my variety of scientific computing (arrays in particular), and it has many compilers. Certainly, a better language could be designed without the legacy baggage and some features I'd like (better generics in particular), but I haven't seen anything better yet myself. Now, legacy FORTRAN (note caps) is often a mess. It wasn't until the Fortran 90 standard that the capitalization changed, and the language changed a lot with that standard as well. I suspect the issues that you're seeing are more from many scientists not modernizing, and not from Fortran in itself.
- piva00 3y ago
  No, not at all, I didn't want to imply that, simply noted that Fortran is still used a lot in science and that scientists would greatly benefit from programmers to mentor them on how to write code in whatever language in a better way.
  - aragilar 3y ago
    https://software-carpentry.org/ https://software-carpentry.org/ has been around for over 10 years. Also, software engineers without the requisite background are usually worse for a project (and for teaching researchers) than anyone else.

by piva00 3y ago

konradha 3y ago

There is project bringing Fortran to the browser, though (_using WebAssembly_) https://dev.lfortran.org/ https://dev.lfortran.org/

mike_hearn 3y ago

A lot of scientific code is worse than a mess, it outright doesn't work and yields incorrect results, a problem which is then routinely covered up. I've seen this first hand :( The worst part is that universities don't realize how much they don't know. On the rare occasions outsiders notice what's going on they are faced with a wall of baffling excuses and justifications for why so much doesn't work, like "if it didn't crash it must be correct" or "scientists don't need unit tests, we just look at the results and know they are right because we're experts". Academics are happy to pronounce that professional coders can't judge the correctness of their work and do so loudly and publicly. As for money, well departments do have the budgets to hire developers. Science funding is in the high billions in most western countries. The problem is not departmental budgets, it's a social problem. Universities love the practice of spreading money amongst as many professors and tiny departments as possible in order to lay claim to every possible area of human knowledge/experience. Combine that with a culture of low standards and coverups and you've got a recipe for disasters (e.g. invariably critical bug fixes that corrupt data are described as not affecting the final results even when that clearly can't be true).

abyssin 3y ago
I have the same experience in a psychology faculty. I helped a PhD student who was working on some MATLAB code. The amount of code I had to write for something similar as they were doing was orders of magnitude lower, and faster to write. But they spent months hurting their brains and therefor thought they were doing something really smart. There was a lot of micromanagement of clueless researchers, and the consequence is that everything had to be done the exact same way as the more experienced members of the lab did previously. This was the most wasteful work organization I've seen in my life, but there was little awareness of it, compared to other state-subsidized projects.
abought 3y ago
Some departments are starting to hire programmers. There's an effort to define this as a (broad) job category under the heading "Research Software Engineer": https://society-rse.org/ https://society-rse.org/ https://us-rse.org/ https://us-rse.org/ Institutional support varies widely; some projects or teams are rather well funded for big projects and senior talent, while at other schools, the cost structure is more aimed at "one off" projects staffed by more recent graduates. A recent grant is trying to fund this work at several schools with a history of well organized services: https://www.schmidtfutures.org/our-work-old/virtual-institute-for-scientific-software/ https://www.schmidtfutures.org/our-work-old/virtual-institut...
- mike_hearn 3y ago
  That's been around for a while but the pay for these roles is very uncompetitive, and universities don't care if professors don't do it. It doesn't make any difference to whether you get published or not so it'll probably remain niche. Now, if journals start rejecting papers because the code isn't provided or because it was provided but professional software engineers reviewed it and gave it a thumbs down, that'd change. But journals struggle to keep obviously AI generated material out of their pages, so they aren't going to do anything like that.
  - abought 3y ago
    It depends. Some of the foundation-funded positions are competitive, and a few centers have surprisingly professional leadership. People are trying to organize, and- even if it's an uphill climb- there's been some improvement at the edges. Anecdotally, some of the RSE leads I've spoken to are seeing more long-term demand than they predicted, which might lead to more room for senior roles. Currently quite a few teams (outside the big centers) seem to be priced way too low, usually explained as because they're testing the waters.... so "cheap student labor" and "one off project" is what they can afford. Minor heretical aside: one thing I miss about old twitter is that academia was developing a real "second layer" on top of journals, where things like reproducibility could be discussed publicly. PubPeer is a partial solution, as are GitHub issues... if enough gatekeepy people really see value in code quality, norms will shift with or without mandates.

btrettel 3y ago

I get the impression that you think that Fortran is a bad language for science. Modern Fortran is a nice language for many scientific computing tasks. For me, I like Fortran because it's basically the easiest statically-typed compiled language (I rarely have to think about pointers, for instance), it's basically as fast as C, it has some good features for my variety of scientific computing (arrays in particular), and it has many compilers. Certainly, a better language could be designed without the legacy baggage and some features I'd like (better generics in particular), but I haven't seen anything better yet myself. Now, legacy FORTRAN (note caps) is often a mess. It wasn't until the Fortran 90 standard that the capitalization changed, and the language changed a lot with that standard as well. I suspect the issues that you're seeing are more from many scientists not modernizing, and not from Fortran in itself.

piva00 3y ago
No, not at all, I didn't want to imply that, simply noted that Fortran is still used a lot in science and that scientists would greatly benefit from programmers to mentor them on how to write code in whatever language in a better way.
- aragilar 3y ago
  https://software-carpentry.org/ https://software-carpentry.org/ has been around for over 10 years. Also, software engineers without the requisite background are usually worse for a project (and for teaching researchers) than anyone else.