3 ms·

>-Is actually familiar with academic code so that the usual pearl clutching can be tempered with realism. I think this is a real and systemic problem. Academia

>-Is actually familiar with academic code so that the usual pearl clutching can be tempered with realism. I think this is a real and systemic problem. Academia has driven itself into this horrible cultural rut where they've defined for themselves this attitude towards code that it's not important and it's fine if it's bad, hacky, undocumented, not version controlled, copied and pasted over and over. Its just research code! It's not pearl clutching to point out that this is the coding version of a teenager saving final_report.docx, final_report_final.docx, final_report_3/23/2020.docx, final_report_v3.docx. It's basically everyone outside of academia looking at academics and wondering why they don't just grow up, and it's a cultural issue. The problem is that what you describe as pearl clutching is just the very reasonable observation that an entire profession has chosen to just refuse to do a core part of their job.

Mvandenbergh 6y ago
The problem is that it is not everyone outside of academia. The vast majority of the world's modelling code for various values of modelling is written and "maintained" by subject matter experts who are not software engineers. Finance, insurance, you name it and there's a horrible Excel model driving it.
- Traster 6y ago
  I think the core difference is that most places outside of academia there is a closed feedback loop. You make a crap prediction, you lose a bunch of money, you get fired. There's a self-correcting mechanism for insuring the models work correctly. In academia you run your model, you write your paper, you publish your paper, you get your promotion. Because no decisions are made based on your model, there's no accountability. Even crappy excel models in business are more reliable than academic ones because people actually have to use the results- and that compounds over decades. But the impact of lots of academics generating rubbish and publishing it is that we're literally polluting our knowledge pool. Which is why it should be of as important for your code to be right as it is for your experimental method to be right.
  - Mvandenbergh 6y ago
    Sure, models that break along their common paths will lose people money. Models that don't produce the results they should won't lead to high impact publications. In both cases, models can break in unfortunate ways along rarely used code paths or under unusual conditions. If a model just doesn't work, then of course it will be fixed. That's in any context. If it works most of the time until it breaks badly once, then it may well not be fixed before it has caused massive damage.

Mvandenbergh 6y ago

The problem is that it is not everyone outside of academia. The vast majority of the world's modelling code for various values of modelling is written and "maintained" by subject matter experts who are not software engineers. Finance, insurance, you name it and there's a horrible Excel model driving it.

Traster 6y ago
I think the core difference is that most places outside of academia there is a closed feedback loop. You make a crap prediction, you lose a bunch of money, you get fired. There's a self-correcting mechanism for insuring the models work correctly. In academia you run your model, you write your paper, you publish your paper, you get your promotion. Because no decisions are made based on your model, there's no accountability. Even crappy excel models in business are more reliable than academic ones because people actually have to use the results- and that compounds over decades. But the impact of lots of academics generating rubbish and publishing it is that we're literally polluting our knowledge pool. Which is why it should be of as important for your code to be right as it is for your experimental method to be right.
- Mvandenbergh 6y ago
  Sure, models that break along their common paths will lose people money. Models that don't produce the results they should won't lead to high impact publications. In both cases, models can break in unfortunate ways along rarely used code paths or under unusual conditions. If a model just doesn't work, then of course it will be fixed. That's in any context. If it works most of the time until it breaks badly once, then it may well not be fixed before it has caused massive damage.