8 ms·

I used to maintain a large FORTRAN codebase - it seemed like every time new grad students showed up, everyone got a wise idea to re-build everything in a modern

I used to maintain a large FORTRAN codebase - it seemed like every time new grad students showed up, everyone got a wise idea to re-build everything in a modern language. This question is revisited a lot, but the general consensus has been that FORTRAN is fast, simple, easy to understand and code, and the compilers used are super optimized (Intel's FORTRAN compiler was really a gem - it managed to do a ton of automatic parallelization on modern hardware) I'll see if I can find it, but I remember attending a conference talk at AGU in ~'08-10 called "Do-over or make-do" [1] that analyzed how many people-hours it would take to update all the climate modeling software (earth and space sciences are really huge on legacy FORTRAN code), and the conclusion of that talk was: use modern code/tools for glue, keep your climate models in FORTRAN. There's a ton relating to precision - we had a huge ordeal converting our 32-bit space weather model to 64-bit because the change in precision changed our results, so we couldn't publish (in good conscious) without making sure the results weren't within a good margin of error. Anyhow. This is always a fun thing to revisit now that I'm really far away from having to maintain any FORTRAN code any more :) edit found it! [1]: https://www.easterbrook.ca/steve/2010/12/agu-session-on-software-engineering-for-climate-modeling/ https://www.easterbrook.ca/steve/2010/12/agu-session-on-soft... [2]: slides http://www.cs.toronto.edu/~sme/presentations/Easterbrook-AGU-fall2010.pdf http://www.cs.toronto.edu/~sme/presentations/Easterbrook-AGU... Also randomly found this: http://www.moreisdifferent.com/2015/07/16/why-physicsts-still-use-fortran/ http://www.moreisdifferent.com/2015/07/16/why-physicsts-stil...

classified 6y ago
I came here to ask whether your typical Fortran program would carry the coding security risks that Rust is designed to mitigate; I couldn't think of any. Your comment seems to confirm my doubts.
- codezero 6y ago
  It probably wouldn't have affected my work at all. We were running our code as a monolith - it was the only thing that ran on any given machine, and its output was stored on a large filesystem on NFS. That's not to say there isn't someone out there with this concern :)
  - mumblemumble 6y ago
    https://www.fortran.io/ https://www.fortran.io/ For example. More seriously, I could imagine a security flaw in numpy wreaking all sorts of havok. But it's also somewhat difficult for me to imagine that there are any security vulnerabilities left in BLAS. But then I remember Heartbleed.
    - veddox 6y ago
      I think „coding security risks“ referred to language properties that increase the likelihood of introducing bugs in a general sense, not in terms of cyber security/hacking.
- bsder 6y ago
  Memory errors also affect correctness. Forgetting to initialize that last entry or initializing one too many entries by accident can cause random, very hard to track down, issues. Security? Perhaps if your code is pulling chunks of data from a cloud source?
  - classified 6y ago
    You are right, I forgot to mention correctness.
_bxg1 6y ago
> we had a huge ordeal converting our 32-bit space weather model to 64-bit because the change in precision changed our results, so we couldn't publish (in good conscious) without making sure the results weren't within a good margin of error This is a really interesting dimension to the transition that I've never considered before. An actual change in correctness, not just compatibility. I would have thought for floats it would just add more decimal points
- codezero 6y ago
  FWIW, I didn't bother getting too deep into an analysis (I was an undergrad and not really savvy enough to deep dive, but maybe I could now if I went back :) ) Our model was a space weather model of the solar wind speed and density - it was very iterative, and I think that iterative nature is what contributed to quirks in the transition - think of the issues when summing say, 0.1 ten times in Python (or other languages) - ultimately, we didn't need to do any real work to show that our converged model was within a margin of error and still presented the same results in terms of predicting solar flares - but it was still a lingering curiosity that the numbers weren't exactly the same (as you say!) - most of the work was done in finding the right compiler parameters to not break existing code, while taking advantage of the larger memory space.
  - _bxg1 6y ago
    Ah that makes sense
- giardini 6y ago
  One of our numerical analysis(NA) professors had worked at NASA, at IBM and had enormous real-world experience in NA. Several weeks into the class he challenged us to write a correct square root routine in FORTRAN. Over the next two weeks submissions were presented in class, where our professsor showed how each had multiple critically fatal errors. He let it stew awhile: two weeks before semester's end he presented and explicated to us a correct implementation of the square root, which turned out to be far more lengthy and involved than we had imagined. He said there was always the possibility of more bugs being present. That was really an eye-opening class.
xioxox 6y ago
Although Fortran compilers may produce fast code (they have to - there's no chance to play with SIMD intrinsics or templates unless you want to use C++), I don't find them very good. I've personally hit several bugs in gfortran. The code I maintain now has some extremely ugly hacks to work around unfixed compiler bugs. I recently tried compiling it with Intel Fortran and got a compiler crash on one input file. Maybe they work well with Fortran 77 level code, but the Fortran 2003 level support seemed rather buggy. I suppose this is the problem with a minority language. I won't recommend Fortran unless the code has to be written by scientists who know nothing else. It has horrible string support, no data structure or algorithm library (like STL), no templates, few non-numeric libraries (e.g. http, json...) and (as above) buggy compilers. Furthermore, C++ gives you much more control over numerical speed when you need it.
- TheRealKing 6y ago
  "It has horrible string support", this is untrue. Learn Fortran 2003, 2008, 2018.
  - xioxox 6y ago
    What? Where is this good string support in Fortran 2003, 2008 and 2018? I must have missed it in the manual. Now you can allocate a string of a non fixed size - simply amazing technology! Actually, one of my compiler crashes was trying to use this revolutionary feature.
    - pantalaimon 6y ago
      > Now you can allocate a string of a non fixed size - simply amazing technology! That’s more than what C gives you out of the box ;)
      - smabie 6y ago
        Many languages forbid mutable strings. They aren't really a very good idea, imo. Even less of a good idea are extensible strings. More complex data structures are required for efficient extensible strings, and this complexity shouldn't be hidden in a simple interface. Better to have a Buffer type or something that is distinct from the String type so programmers are made aware of the performance/allocation characteristics. Though, this is kind of ridiculous to even talk about, since the CPU and compiler itself are responsible for the greatest sins of hiding something unimaginably complex behind a simple interface. Ah, the joys of the 6502 and the Commodore 64!
nahuel0x 6y ago
Julia is taking some inroads in climate modelling: "when the coder-climatologists of the Climate Modeling Alliance (CliMA) — a coalition of US-based scientists, engineers and mathematicians — set out to build a model from the ground up, they opted for a language that could handle their needs. They opted for Julia." https://www.nature.com/articles/d41586-019-02310-3 https://www.nature.com/articles/d41586-019-02310-3 https://clima.caltech.edu/ https://clima.caltech.edu/
- codezero 6y ago
  That's awesome, and not at all a surprise! It makes a lot of sense when building from the ground up to choose a great tool, and Julia has had some serious adoption in academia.
  - Certhas 6y ago
    I think long term it has enormous potential to take over as _the_ language for modelling. The reason is that for every hyper optimized climate model there are a thousand small or medium models. For these Julia offers sufficient performance and excellent developer experience.
- cjhanks 6y ago
  It probably helps that Julia and Fortran are both column major. When interfacing with C languages, that can be a big penalty (depending on the complexity of the algorithm). It will probably take two generations of developers to make this happen, but well worth it. When I was studying atmospheric science I tried to understand OpenWRF (weather research forecasting) and the barrier was too high... And I had programming experience, imagine those with none! An open source project is only as useful as it is accessible to it's average developer base.
Uhhrrr 6y ago
"in good conscience"
- codezero 6y ago
  Nice catch, thanks!
MaxBarraclough 6y ago
> There's a ton relating to precision - we had a huge ordeal converting our 32-bit space weather model to 64-bit because the change in precision changed our results, so we couldn't publish (in good conscious) without making sure the results weren't within a good margin of error. That sounds like a question of numerical stability of the algorithm, rather than anything to do with the programming language, no?
- codezero 6y ago
  Yep. Fair point. The folks I worked with were very pragmatic so didn’t get into the internals and just treated the code as the instructions. Many of them came from an earlier era where the code was a lot more specific and the concept of an R8 is not quite as specific on x86_84 as it was on a VAX because the modern CPU does a lot of tricks under your nose.
lmm 6y ago
> we had a huge ordeal converting our 32-bit space weather model to 64-bit because the change in precision changed our results, so we couldn't publish (in good conscious) without making sure the results weren't within a good margin of error. Had you done some kind of external validation of the original results? If not, surely the new results are just as valid to publish as the original ones - any bugs are as much or more likely to be in the 32-bit than the 64-bit version.
- codezero 6y ago
  Yep. Our model was run by other organizations and incorporated into amalgamated models that had further downstream error bars that they needed. Not that our model contributed much but we were one tiny part of the early warning system for solar flares for satellites and astronauts. We admittedly weren’t super formal about the transition but wanted to make sure the differences wouldn’t be meaningful or propagate into other models we contributed to. Also fwiw, we had no control or say over how others ran our model so they could have easily done their own 64 bit build of our code but I think it’s unlikely.

classified 6y ago

I came here to ask whether your typical Fortran program would carry the coding security risks that Rust is designed to mitigate; I couldn't think of any. Your comment seems to confirm my doubts.

codezero 6y ago
It probably wouldn't have affected my work at all. We were running our code as a monolith - it was the only thing that ran on any given machine, and its output was stored on a large filesystem on NFS. That's not to say there isn't someone out there with this concern :)
- mumblemumble 6y ago
  https://www.fortran.io/ https://www.fortran.io/ For example. More seriously, I could imagine a security flaw in numpy wreaking all sorts of havok. But it's also somewhat difficult for me to imagine that there are any security vulnerabilities left in BLAS. But then I remember Heartbleed.
  - veddox 6y ago
    I think „coding security risks“ referred to language properties that increase the likelihood of introducing bugs in a general sense, not in terms of cyber security/hacking.
bsder 6y ago
Memory errors also affect correctness. Forgetting to initialize that last entry or initializing one too many entries by accident can cause random, very hard to track down, issues. Security? Perhaps if your code is pulling chunks of data from a cloud source?
- classified 6y ago
  You are right, I forgot to mention correctness.

_bxg1 6y ago

> we had a huge ordeal converting our 32-bit space weather model to 64-bit because the change in precision changed our results, so we couldn't publish (in good conscious) without making sure the results weren't within a good margin of error This is a really interesting dimension to the transition that I've never considered before. An actual change in correctness, not just compatibility. I would have thought for floats it would just add more decimal points

codezero 6y ago
FWIW, I didn't bother getting too deep into an analysis (I was an undergrad and not really savvy enough to deep dive, but maybe I could now if I went back :) ) Our model was a space weather model of the solar wind speed and density - it was very iterative, and I think that iterative nature is what contributed to quirks in the transition - think of the issues when summing say, 0.1 ten times in Python (or other languages) - ultimately, we didn't need to do any real work to show that our converged model was within a margin of error and still presented the same results in terms of predicting solar flares - but it was still a lingering curiosity that the numbers weren't exactly the same (as you say!) - most of the work was done in finding the right compiler parameters to not break existing code, while taking advantage of the larger memory space.
- _bxg1 6y ago
  Ah that makes sense
giardini 6y ago
One of our numerical analysis(NA) professors had worked at NASA, at IBM and had enormous real-world experience in NA. Several weeks into the class he challenged us to write a correct square root routine in FORTRAN. Over the next two weeks submissions were presented in class, where our professsor showed how each had multiple critically fatal errors. He let it stew awhile: two weeks before semester's end he presented and explicated to us a correct implementation of the square root, which turned out to be far more lengthy and involved than we had imagined. He said there was always the possibility of more bugs being present. That was really an eye-opening class.

xioxox 6y ago

Although Fortran compilers may produce fast code (they have to - there's no chance to play with SIMD intrinsics or templates unless you want to use C++), I don't find them very good. I've personally hit several bugs in gfortran. The code I maintain now has some extremely ugly hacks to work around unfixed compiler bugs. I recently tried compiling it with Intel Fortran and got a compiler crash on one input file. Maybe they work well with Fortran 77 level code, but the Fortran 2003 level support seemed rather buggy. I suppose this is the problem with a minority language. I won't recommend Fortran unless the code has to be written by scientists who know nothing else. It has horrible string support, no data structure or algorithm library (like STL), no templates, few non-numeric libraries (e.g. http, json...) and (as above) buggy compilers. Furthermore, C++ gives you much more control over numerical speed when you need it.

TheRealKing 6y ago
"It has horrible string support", this is untrue. Learn Fortran 2003, 2008, 2018.
- xioxox 6y ago
  What? Where is this good string support in Fortran 2003, 2008 and 2018? I must have missed it in the manual. Now you can allocate a string of a non fixed size - simply amazing technology! Actually, one of my compiler crashes was trying to use this revolutionary feature.
  - pantalaimon 6y ago
    > Now you can allocate a string of a non fixed size - simply amazing technology! That’s more than what C gives you out of the box ;)
    - smabie 6y ago
      Many languages forbid mutable strings. They aren't really a very good idea, imo. Even less of a good idea are extensible strings. More complex data structures are required for efficient extensible strings, and this complexity shouldn't be hidden in a simple interface. Better to have a Buffer type or something that is distinct from the String type so programmers are made aware of the performance/allocation characteristics. Though, this is kind of ridiculous to even talk about, since the CPU and compiler itself are responsible for the greatest sins of hiding something unimaginably complex behind a simple interface. Ah, the joys of the 6502 and the Commodore 64!

nahuel0x 6y ago

Julia is taking some inroads in climate modelling: "when the coder-climatologists of the Climate Modeling Alliance (CliMA) — a coalition of US-based scientists, engineers and mathematicians — set out to build a model from the ground up, they opted for a language that could handle their needs. They opted for Julia." https://www.nature.com/articles/d41586-019-02310-3 https://www.nature.com/articles/d41586-019-02310-3 https://clima.caltech.edu/ https://clima.caltech.edu/

codezero 6y ago
That's awesome, and not at all a surprise! It makes a lot of sense when building from the ground up to choose a great tool, and Julia has had some serious adoption in academia.
- Certhas 6y ago
  I think long term it has enormous potential to take over as _the_ language for modelling. The reason is that for every hyper optimized climate model there are a thousand small or medium models. For these Julia offers sufficient performance and excellent developer experience.
cjhanks 6y ago
It probably helps that Julia and Fortran are both column major. When interfacing with C languages, that can be a big penalty (depending on the complexity of the algorithm). It will probably take two generations of developers to make this happen, but well worth it. When I was studying atmospheric science I tried to understand OpenWRF (weather research forecasting) and the barrier was too high... And I had programming experience, imagine those with none! An open source project is only as useful as it is accessible to it's average developer base.

Uhhrrr 6y ago

"in good conscience"

codezero 6y ago
Nice catch, thanks!

MaxBarraclough 6y ago

> There's a ton relating to precision - we had a huge ordeal converting our 32-bit space weather model to 64-bit because the change in precision changed our results, so we couldn't publish (in good conscious) without making sure the results weren't within a good margin of error. That sounds like a question of numerical stability of the algorithm, rather than anything to do with the programming language, no?

codezero 6y ago
Yep. Fair point. The folks I worked with were very pragmatic so didn’t get into the internals and just treated the code as the instructions. Many of them came from an earlier era where the code was a lot more specific and the concept of an R8 is not quite as specific on x86_84 as it was on a VAX because the modern CPU does a lot of tricks under your nose.

lmm 6y ago

> we had a huge ordeal converting our 32-bit space weather model to 64-bit because the change in precision changed our results, so we couldn't publish (in good conscious) without making sure the results weren't within a good margin of error. Had you done some kind of external validation of the original results? If not, surely the new results are just as valid to publish as the original ones - any bugs are as much or more likely to be in the 32-bit than the 64-bit version.

codezero 6y ago
Yep. Our model was run by other organizations and incorporated into amalgamated models that had further downstream error bars that they needed. Not that our model contributed much but we were one tiny part of the early warning system for solar flares for satellites and astronauts. We admittedly weren’t super formal about the transition but wanted to make sure the differences wouldn’t be meaningful or propagate into other models we contributed to. Also fwiw, we had no control or say over how others ran our model so they could have easily done their own 64 bit build of our code but I think it’s unlikely.