3 ms·

You know more about the subject than I do, but I don't think you're entirely contradicting one of the author's key points: It's incredibly hard to rewrite a sys

You know more about the subject than I do, but I don't think you're entirely contradicting one of the author's key points: It's incredibly hard to rewrite a system from scratch - and in effect, to just write a system from scratch. But it would make sense to have a "professional scientific fortran programmer" go in and do some housecleaning to make things right, without whining, "Why can't I just rewrite it in Java???" Heck, it sounds like fun to me, but that's because I'm basically a professional janitor, and hey, hurricanes and polar vortexes and all that. So why doesn't that happen? Is testing improvements too expensive? Do scientific programmers just have no respect for the craft whatsoever? Is there nobody with the necessary interdisciplinary skill set?

hfbff 4y ago
My experience with academics is that most don't know / don't understand the benefits of coding things in a better way, so they don't even consider hiring someone to help. The other issue is that the incentives aren't necessarily aligned. They don't get payed to share their code or to make their code reusable. The code is used to publish one or a couple of paper and then thrown away. Maybe someone else will reuse it in the future but the original author doesn't necessarily gain much from that
machina_ex_deus 4y ago
The way I see it is analogically to writing secure and memory safe code before people knew these things matter. You needed someone with an incentive to go over code, find the bugs, break into computers and for the entire industry to mature from using strcat to having bug bounties and fuzzing and protect zero. It didn't happen over night and there had to be incredible incentives on the attacker / bug finder to exist, to probe and test the parts that aren't tested, and to serve as the devil's advocate. The analogy is that it is hard to find bugs which happen silently. Computation inaccuracies are a huge class of bugs that also happen silently. There are no incentives to look at old code. There's just nothing like it for climate models, and the default behavior of code development is that whatever isn't tested or validated by something will always stagnate with bugs. The code copy just makes everything worse because implementing the same thing in two different ways by different people is actually good at catching bugs. So even if I wouldn't have trusted a single climate model, I might trust several climate models developed independently. The way it works now is that they always prefer to rely on code of others, because academia is driven by citations, and creating new models from scratch means you're missing out mutual citations. It's a systematic problem, the same way security had been a systemic problem: current methodology just doesn't produce code that fits the requirements. Nobody had neither the incentive nor even the idea that these things should work differently. Admitting that this is a systemic problem is just a politicized subject. There are plenty of people saying these models are problematic but getting those working on the models to admit that enough such that the entire system can change its methodology, that won't happen because of human reasons.
- kerblang 4y ago
  > The way it works now is that they always prefer to rely on code of others, because academia is driven by citations, and creating new models from scratch means you're missing out mutual citations. Ah! This is the interesting point that stands out, as I hadn't anticipated such an incentive. It's a nasty one.
tored 4y ago
I guess that you need at least some scientific knowledge about the project's subject in order to work with the domain and maintain the code, but not on an expert level, because that would be a waste of resources put a great scientist on maintaining and polishing old code (even if it is important). And why would a good programmer agree to suffer to the whims of a Professor that is clueless about programming even if the domain itself is important and interesting?
lmm 4y ago
> So why doesn't that happen? Is testing improvements too expensive? Do scientific programmers just have no respect for the craft whatsoever? Is there nobody with the necessary interdisciplinary skill set? Academia relies on paying grad students peanuts and dangling the carrot of being able to get citations under their name to make up for it. There's no way to publish a paper by cleaning up code, much less get cited for doing so. Bear in mind there's basically zero "general" funding available. Everything has to be paid out of a specific grant and they want to see it being used for that specific purpose, which likely ends with publishing a paper. But the short version is yes, academia has no respect for the craft. Papers are what matters, so code will always be the minimum necessary to publish a paper, which is generally: copy someone else's fortran/numpy code, tweak it enough until it runs (once, on your laptop) and gives the result you were looking for, publish.
mamcx 4y ago
> "professional scientific fortran programmer" That is too late You CAN'T start with a developer that, hopefully, do good code. You MUST make good code to be the default, from the start. Is the only chance of lasting improvements. ie: Is necessary a "Rust/Ada for Fortran", and this ALSO must make, somehow, make some significant impact in current Fortran developers, make them salivate for the chance of do the change. But iterate on Fortran(C/C++..) and hope that will be better? NEVER will work.

hfbff 4y ago

My experience with academics is that most don't know / don't understand the benefits of coding things in a better way, so they don't even consider hiring someone to help. The other issue is that the incentives aren't necessarily aligned. They don't get payed to share their code or to make their code reusable. The code is used to publish one or a couple of paper and then thrown away. Maybe someone else will reuse it in the future but the original author doesn't necessarily gain much from that

machina_ex_deus 4y ago

The way I see it is analogically to writing secure and memory safe code before people knew these things matter. You needed someone with an incentive to go over code, find the bugs, break into computers and for the entire industry to mature from using strcat to having bug bounties and fuzzing and protect zero. It didn't happen over night and there had to be incredible incentives on the attacker / bug finder to exist, to probe and test the parts that aren't tested, and to serve as the devil's advocate. The analogy is that it is hard to find bugs which happen silently. Computation inaccuracies are a huge class of bugs that also happen silently. There are no incentives to look at old code. There's just nothing like it for climate models, and the default behavior of code development is that whatever isn't tested or validated by something will always stagnate with bugs. The code copy just makes everything worse because implementing the same thing in two different ways by different people is actually good at catching bugs. So even if I wouldn't have trusted a single climate model, I might trust several climate models developed independently. The way it works now is that they always prefer to rely on code of others, because academia is driven by citations, and creating new models from scratch means you're missing out mutual citations. It's a systematic problem, the same way security had been a systemic problem: current methodology just doesn't produce code that fits the requirements. Nobody had neither the incentive nor even the idea that these things should work differently. Admitting that this is a systemic problem is just a politicized subject. There are plenty of people saying these models are problematic but getting those working on the models to admit that enough such that the entire system can change its methodology, that won't happen because of human reasons.

kerblang 4y ago
> The way it works now is that they always prefer to rely on code of others, because academia is driven by citations, and creating new models from scratch means you're missing out mutual citations. Ah! This is the interesting point that stands out, as I hadn't anticipated such an incentive. It's a nasty one.

tored 4y ago

I guess that you need at least some scientific knowledge about the project's subject in order to work with the domain and maintain the code, but not on an expert level, because that would be a waste of resources put a great scientist on maintaining and polishing old code (even if it is important). And why would a good programmer agree to suffer to the whims of a Professor that is clueless about programming even if the domain itself is important and interesting?

lmm 4y ago

> So why doesn't that happen? Is testing improvements too expensive? Do scientific programmers just have no respect for the craft whatsoever? Is there nobody with the necessary interdisciplinary skill set? Academia relies on paying grad students peanuts and dangling the carrot of being able to get citations under their name to make up for it. There's no way to publish a paper by cleaning up code, much less get cited for doing so. Bear in mind there's basically zero "general" funding available. Everything has to be paid out of a specific grant and they want to see it being used for that specific purpose, which likely ends with publishing a paper. But the short version is yes, academia has no respect for the craft. Papers are what matters, so code will always be the minimum necessary to publish a paper, which is generally: copy someone else's fortran/numpy code, tweak it enough until it runs (once, on your laptop) and gives the result you were looking for, publish.

mamcx 4y ago

> "professional scientific fortran programmer" That is too late You CAN'T start with a developer that, hopefully, do good code. You MUST make good code to be the default, from the start. Is the only chance of lasting improvements. ie: Is necessary a "Rust/Ada for Fortran", and this ALSO must make, somehow, make some significant impact in current Fortran developers, make them salivate for the chance of do the change. But iterate on Fortran(C/C++..) and hope that will be better? NEVER will work.