4 ms·

I've spent years learning and practicing good software engineering practices. Whenever I read about quality issues in scientific code, I wonder: how are they e

I've spent years learning and practicing good software engineering practices. Whenever I read about quality issues in scientific code, I wonder: how are they expected to learn these things?

searchableguy 6y ago
I don't think you need to worry about more than a few principles - 1. Reproducibility This is very important for scientists so I expect them to know about package management and containers. You don't need to know everything about it. Virtual environments, docker and using a good package manager like poetry (which is just an abstraction over virtualenv) shouldn't take more than a few weeks. Once you are done with the basics, you can learn more as you build. 2. Git and documentation Add another few weeks. They don't need to write good code. They just need to document it which they should already be good at. Optimization, scalability, distribution and other things can be handled by engineers but without documentation, it makes the work a lot harder.
- sgt101 6y ago
  Every engineering team that picked up a code base has at some point had a meeting where someone starts ranting about how the code that they are looking after stinks and it would be better to rewrite it, or move to some alternative. This seems to be true if the code is the product of one person working for a few weeks or 20,000 people who work for 10 years. A few years ago I had a conversation where the team insisted to me that the extensive documentation and instructions for a package were worthless because "good code is self documenting" the leader of the pack had shiny eyes and banged the table several times. Be cautious, be kind and look for value in the code that you are using - not just problems. Also, consider: if you can pick up a code base (written by someone who knows the science) would it not be more profitable to repair, document and improve rather than demanding that the scientists learn both your skills and your way of doing things? If you conclude the latter, would you (hand on heart) be able to empirically demonstrate that your way of doing things is the right way?
- throwaway_pdp09 6y ago
  I disagree with "They don't need to write good code". Reproducability is a check that you got the right answer. As you describe it, it's more like a crappy programmer that relies on the testing team to catch his errors rather than be responsible for his own. > and other things can be handled by engineers Due to lack of funding and low pay I think most of them don't have access to software domain experts (aka seasoned programmers)
yummypaint 6y ago
Physicist here. The lack of any formal training in most programs is a problem. In 2020 it's essentially impossible to do research without being able to write and maintain code, yet there are no real standards of knowledge for the dicipline as a whole. I dont know of any undergrad programs requiring physics majors to take even intro to cs. Most incoming grad students have never used version control, and probably about half of professors have never used it, let alone want to start using it. Larger collaborations tend to be more rigorous with coding practices because they have dedicated software engineers who enforce order and show people the way. Small groups with a few students and a professor are basically the wild west as far as writing software, and vary wildly. In practice most grad students get coding experience by working on projects, which is a great way to learn, but its rare for anyone to look at the code itself afterward if the results are sensible. I dont know of any small group with a formal code review process. I think we could make relatively inexpensive improvements in science by bringing in more dedicated software people who can educate others and help with the software engineering aspects. Its not unusual for a lab with half a dozen groups and 50+ scientists to have no software specialists, and 1 or 2 IT people whose responsibilities are purely infrastructure oriented.
dekhn 6y ago
This is the core question. Can you teach good software engineering, or does it come from years of working with more experienced people/making mistakes and learning? I have met a few "born SREs" who had little or no training but knew exactly what to do in production outage situations. But I haven't met anybody who was a "Great Software Engineer" out of the box. And scientists aren't going to take a lot of time out of their education to get that day-to-day programming experience. I was lucky- I worked in a theory lab, next to a team of good software developers, and worked/learned from them over a period of years. I think what we need are more research engineers, who are people who do software engineering for scientists, so the scientists don't have to become software engineering experts. The scientists also need to learn a bit better to ask for the right tools.

I've spent years learning and practicing good software engineering practices. Whenever I read about quality issues in scientific code, I wonder: how are they expected to learn these things?

searchableguy 6y ago

I don't think you need to worry about more than a few principles - 1. Reproducibility This is very important for scientists so I expect them to know about package management and containers. You don't need to know everything about it. Virtual environments, docker and using a good package manager like poetry (which is just an abstraction over virtualenv) shouldn't take more than a few weeks. Once you are done with the basics, you can learn more as you build. 2. Git and documentation Add another few weeks. They don't need to write good code. They just need to document it which they should already be good at. Optimization, scalability, distribution and other things can be handled by engineers but without documentation, it makes the work a lot harder.

sgt101 6y ago
Every engineering team that picked up a code base has at some point had a meeting where someone starts ranting about how the code that they are looking after stinks and it would be better to rewrite it, or move to some alternative. This seems to be true if the code is the product of one person working for a few weeks or 20,000 people who work for 10 years. A few years ago I had a conversation where the team insisted to me that the extensive documentation and instructions for a package were worthless because "good code is self documenting" the leader of the pack had shiny eyes and banged the table several times. Be cautious, be kind and look for value in the code that you are using - not just problems. Also, consider: if you can pick up a code base (written by someone who knows the science) would it not be more profitable to repair, document and improve rather than demanding that the scientists learn both your skills and your way of doing things? If you conclude the latter, would you (hand on heart) be able to empirically demonstrate that your way of doing things is the right way?
throwaway_pdp09 6y ago
I disagree with "They don't need to write good code". Reproducability is a check that you got the right answer. As you describe it, it's more like a crappy programmer that relies on the testing team to catch his errors rather than be responsible for his own. > and other things can be handled by engineers Due to lack of funding and low pay I think most of them don't have access to software domain experts (aka seasoned programmers)

yummypaint 6y ago

Physicist here. The lack of any formal training in most programs is a problem. In 2020 it's essentially impossible to do research without being able to write and maintain code, yet there are no real standards of knowledge for the dicipline as a whole. I dont know of any undergrad programs requiring physics majors to take even intro to cs. Most incoming grad students have never used version control, and probably about half of professors have never used it, let alone want to start using it. Larger collaborations tend to be more rigorous with coding practices because they have dedicated software engineers who enforce order and show people the way. Small groups with a few students and a professor are basically the wild west as far as writing software, and vary wildly. In practice most grad students get coding experience by working on projects, which is a great way to learn, but its rare for anyone to look at the code itself afterward if the results are sensible. I dont know of any small group with a formal code review process. I think we could make relatively inexpensive improvements in science by bringing in more dedicated software people who can educate others and help with the software engineering aspects. Its not unusual for a lab with half a dozen groups and 50+ scientists to have no software specialists, and 1 or 2 IT people whose responsibilities are purely infrastructure oriented.

dekhn 6y ago

This is the core question. Can you teach good software engineering, or does it come from years of working with more experienced people/making mistakes and learning? I have met a few "born SREs" who had little or no training but knew exactly what to do in production outage situations. But I haven't met anybody who was a "Great Software Engineer" out of the box. And scientists aren't going to take a lot of time out of their education to get that day-to-day programming experience. I was lucky- I worked in a theory lab, next to a team of good software developers, and worked/learned from them over a period of years. I think what we need are more research engineers, who are people who do software engineering for scientists, so the scientists don't have to become software engineering experts. The scientists also need to learn a bit better to ask for the right tools.