4 ms·

You often run into code of the "just get it to work" variety, which has the problem that when it was written, maintainability was bottom of the list of prioriti

You often run into code of the "just get it to work" variety, which has the problem that when it was written, maintainability was bottom of the list of priorities. Often the author has a goal that isn't described in terms of software engineering terms: calculate my option model, work out the hedge amounts, etc. And the people who write this kind of code tend not to think about version control, documentation, dependency management, deployment, and so forth. The result is you get these fragile pieces holding up some very complex logic, which takes a lot of effort to understand. IMO there should be a sort of code literacy course that everyone who writes anything needs to do. In a way it's the modern equivalent of everyone who writes needing to understand not just grammar but style and other writing related hygiene.

throwanem 6y ago
The fundamental problem here, as you note, is that scientists are rarely also engineers, and don't really share our desiderata. The point is to develop and publish a result, and engineering analysis code for resiliency is of secondary concern at best when that code isn't likely to need to be used again once the paper is finished. The "Software Carpentry" movement [1] has in the past decade tried to address this, as I recall. It's very much in the vein of the "basic literacy" course you suggest. I can't say how far they've gotten, and I'm no longer adjacent to academia, but based on what I do still see of academics' code, there's a long way still to go. [1] https://software-carpentry.org/ https://software-carpentry.org/
- detaro 6y ago
  And that scientists also are rarely supported by programmers, or if they are it's an unstable and unappreciated position.
  - non-entity 6y ago
    Ive seen job listings for "scientific programmers" where what they're asking for is a scientist who happens to know a little programming.
    - detaro 6y ago
      Yeah - who then likely doesn't have that much software experience, and worse, if they want to stay a scientist such a role is often a bad career move, because they help others get ahead with their research instead of publishing their own work. Even if they build some really great domain-specific software tool in that role, it often doesn't count as much. Or it's an informal thing done by some student as a side-gig. Which can be cool, but is not a stable long-term thing. I hope there's exceptions. EDIT: weirdest example I've seen was a lab looking for sysadmins with PhD preferred. I wonder if they had some funding source that only paid for "scientists" or what was going on there...
      - mnw21cam 6y ago
        Simple answer for that. University pay scales tend to be fairly inflexible in terms of which grades you are eligible for without a PhD, if you are counted as academic staff. If you're non-academic staff (like the cleaner, the receptionist, and the central IT sysadmin) then you can be paid a fair wage based upon your experience, but if you are academic staff, then you have a hard ceiling without a PhD. An individual research group with a grant may only be able to hire academic staff, but they want a sysadmin, so in order to be able to pay them more than a pittance they would have to have a PhD.
  - throwanem 6y ago
    Having had that exact experience - yeah, that can be a big problem too. Researchers and engineers can work really well together, because the strengths of each role complement the weaknesses of the other, and I think it would be very nice to see that actually happen some day.
    - detaro 6y ago
      It doesn't help with the issue of hard-to-reproduce work, but apparently working for a company making products aimed at scientists can be a place to see this happen (if the company is good about talking to customers).
      - throwanem 6y ago
        Interesting, thanks! I'll keep that in mind for when I'm next looking for a new client.
  - mnw21cam 6y ago
    Being in such a position, I can say that I am appreciated, but not in a manner that results in job stability and promotion. It's a massive problem in academia, and there's an attempt to get the position recognised and call it "Research Software Engineer", with comparable opportunities for promotion and job stability as a researcher. However, it's not going massively well. Academic job progression is still almost completely purely based on the ability to get first or last author papers in top journals. I have lots of papers where I am a middle author, because I wrote the software that did the analysis that was vital for the paper to even exist, but it largely doesn't count. And I'm lucky - many software engineers don't even get put in as a middle author on the paper they contributed to.
- neutronicus 6y ago
  Nah. The fundamental problem is that scientific code is produced by entry-level developers: 1. Paid below-market wages 2. With no way to move up in the organization 3. With lots of non-software responsibilities 4. With an expectation of leaving the organization in six years As long as the grunt work of science is done by overworked junior scientists whose careers get thrown to the wolves no matter what they do, you're not going to get maintainable code out of it.
  - throwanem 6y ago
    I mean, senior researchers in stable roles don't really do any better. Just to pick the first example off the top of my head - one of the investigators I worked with, during my year as a staff member of an academic institution most of a decade ago, is also one of my oldest friends; he's been a researcher there for what must be well past ten years by now. Despite one of his undergrad degrees being actually in CS, I still find ample reason whenever I see it to give him a hard time about the maintainability of his code. Like I said before, it's a field in which people really just don't give a damn about engineering. Which is fair! There's little reason why they should, as far as I've ever been able to see.
  - jnxx 6y ago
    Even more fundamental is that there is no maintenance budget for important scientific libraries and tools. Somebody wrote them as part of their job, and the person who wrote it, is now working somewhere else.
justinmeiners 6y ago
Unit testing, readability version control, documentation, etc are all engineering practices for the purpose of making ongoing development organized (especially for teams). Why would a researcher need to do this, when in most cases all that they use is the output, and in CS/math it's only a minimal prototype demonstrating operation of their principle? All of the other stuff would certainly be nice, but they don't need to adopt our whole profession to write code
dhosek 6y ago
Even with all the best practices, things outside your control can cause issues. A lot of the code that software engineers write is subject to tiny bits of continual maintenance as small changes in the runtime environment take place. Imagine ten years of those changes deployed all at once. Even something employing all the best practices of ten years ago could be a challenge. You've got a subversion repository somewhere with the code which was compiled to run on Windows XP with Windows Visual Studio C++ 2008 Express but you've abandoned Windows for Linux. If you're lucky the code will compile with the appropriate flags to support C++98 in gcc, but who knows? And maybe there's a bunch of graphical stuff that isn't supported at all anymore or a computational library you used which was only distributed as a closed-source library for 32-bit Windows.

throwanem 6y ago

The fundamental problem here, as you note, is that scientists are rarely also engineers, and don't really share our desiderata. The point is to develop and publish a result, and engineering analysis code for resiliency is of secondary concern at best when that code isn't likely to need to be used again once the paper is finished. The "Software Carpentry" movement [1] has in the past decade tried to address this, as I recall. It's very much in the vein of the "basic literacy" course you suggest. I can't say how far they've gotten, and I'm no longer adjacent to academia, but based on what I do still see of academics' code, there's a long way still to go. [1] https://software-carpentry.org/ https://software-carpentry.org/

detaro 6y ago
And that scientists also are rarely supported by programmers, or if they are it's an unstable and unappreciated position.
- non-entity 6y ago
  Ive seen job listings for "scientific programmers" where what they're asking for is a scientist who happens to know a little programming.
  - detaro 6y ago
    Yeah - who then likely doesn't have that much software experience, and worse, if they want to stay a scientist such a role is often a bad career move, because they help others get ahead with their research instead of publishing their own work. Even if they build some really great domain-specific software tool in that role, it often doesn't count as much. Or it's an informal thing done by some student as a side-gig. Which can be cool, but is not a stable long-term thing. I hope there's exceptions. EDIT: weirdest example I've seen was a lab looking for sysadmins with PhD preferred. I wonder if they had some funding source that only paid for "scientists" or what was going on there...
    - mnw21cam 6y ago
      Simple answer for that. University pay scales tend to be fairly inflexible in terms of which grades you are eligible for without a PhD, if you are counted as academic staff. If you're non-academic staff (like the cleaner, the receptionist, and the central IT sysadmin) then you can be paid a fair wage based upon your experience, but if you are academic staff, then you have a hard ceiling without a PhD. An individual research group with a grant may only be able to hire academic staff, but they want a sysadmin, so in order to be able to pay them more than a pittance they would have to have a PhD.
- throwanem 6y ago
  Having had that exact experience - yeah, that can be a big problem too. Researchers and engineers can work really well together, because the strengths of each role complement the weaknesses of the other, and I think it would be very nice to see that actually happen some day.
  - detaro 6y ago
    It doesn't help with the issue of hard-to-reproduce work, but apparently working for a company making products aimed at scientists can be a place to see this happen (if the company is good about talking to customers).
    - throwanem 6y ago
      Interesting, thanks! I'll keep that in mind for when I'm next looking for a new client.
- mnw21cam 6y ago
  Being in such a position, I can say that I am appreciated, but not in a manner that results in job stability and promotion. It's a massive problem in academia, and there's an attempt to get the position recognised and call it "Research Software Engineer", with comparable opportunities for promotion and job stability as a researcher. However, it's not going massively well. Academic job progression is still almost completely purely based on the ability to get first or last author papers in top journals. I have lots of papers where I am a middle author, because I wrote the software that did the analysis that was vital for the paper to even exist, but it largely doesn't count. And I'm lucky - many software engineers don't even get put in as a middle author on the paper they contributed to.
neutronicus 6y ago
Nah. The fundamental problem is that scientific code is produced by entry-level developers: 1. Paid below-market wages 2. With no way to move up in the organization 3. With lots of non-software responsibilities 4. With an expectation of leaving the organization in six years As long as the grunt work of science is done by overworked junior scientists whose careers get thrown to the wolves no matter what they do, you're not going to get maintainable code out of it.
- throwanem 6y ago
  I mean, senior researchers in stable roles don't really do any better. Just to pick the first example off the top of my head - one of the investigators I worked with, during my year as a staff member of an academic institution most of a decade ago, is also one of my oldest friends; he's been a researcher there for what must be well past ten years by now. Despite one of his undergrad degrees being actually in CS, I still find ample reason whenever I see it to give him a hard time about the maintainability of his code. Like I said before, it's a field in which people really just don't give a damn about engineering. Which is fair! There's little reason why they should, as far as I've ever been able to see.
- jnxx 6y ago
  Even more fundamental is that there is no maintenance budget for important scientific libraries and tools. Somebody wrote them as part of their job, and the person who wrote it, is now working somewhere else.

justinmeiners 6y ago

Unit testing, readability version control, documentation, etc are all engineering practices for the purpose of making ongoing development organized (especially for teams). Why would a researcher need to do this, when in most cases all that they use is the output, and in CS/math it's only a minimal prototype demonstrating operation of their principle? All of the other stuff would certainly be nice, but they don't need to adopt our whole profession to write code

dhosek 6y ago

Even with all the best practices, things outside your control can cause issues. A lot of the code that software engineers write is subject to tiny bits of continual maintenance as small changes in the runtime environment take place. Imagine ten years of those changes deployed all at once. Even something employing all the best practices of ten years ago could be a challenge. You've got a subversion repository somewhere with the code which was compiled to run on Windows XP with Windows Visual Studio C++ 2008 Express but you've abandoned Windows for Linux. If you're lucky the code will compile with the appropriate flags to support C++98 in gcc, but who knows? And maybe there's a bunch of graphical stuff that isn't supported at all anymore or a computational library you used which was only distributed as a closed-source library for 32-bit Windows.