5 ms·

Working in research as a software engineer can be very frustrating. I wrote the majority of the analysis code for one of the chapters of the IPCC 5th Assessment

Working in research as a software engineer can be very frustrating. I wrote the majority of the analysis code for one of the chapters of the IPCC 5th Assessment and yet was told my name couldn't be put anywhere as I wasn't a 'scientist'. I also consistently see people who have never written much code put in charge of projects where the majority of the work/research is writing software. These projects end up with no code repository use, poor quality control, zero testing, limited OO use - all the things that a professional would take for granted. Worse, they don't perceive the difference between a postdoc who can write code and a professional developer with years of experience. So they hire the cheaper option and end up with the usual 'ball of mud'.

tensor 11y ago
Some would argue that no OO use is a good thing (assuming you are referring to object oriented programming). Expanding on this, writing research code is quite a bit different than writing production code. In particular, requirements can change quickly and drastically. This means that very open designs where you don't hide data (e.g. as is common in OO programming) are often the best designs.
- wtf23737 11y ago
  just no.
  - camkego 11y ago
    This.
- pathelectronica 11y ago
  > requirements can change quickly and drastically the implication that this threat is not present for production code is completely ridiculous.
- mbreese 11y ago
  Research code is production code and should be treated as such. It doesn't matter if requirements change quickly, you still need to treat it like it's production code... you document, test, and deploy it. And you should include the ability to adapt the code to flexible requirements in your initial design. Just because you may only use the program once doesn't mean that you can neglect good software engineering practices. I run into bad code all of the time (some of it mine!) in our research, and it's very frustrating.
- Retric 11y ago
  If your research code is a big ball of mud then all it's results are meaningless. And all your research should be rejected by any competent journal. Sadly, journals tend to ignore such huge issues. But, hey it's not like reproduceable results are important in science or anything.
  - mbreese 11y ago
    The more computational journals are keeping track of this. I've had a manuscript returned for revision because I didn't have enough unit testing in the code. Wet-lab journals, on the other hand, probably let a lot of stuff go on the computation and statistics than they should.
- magicbuzz 11y ago
  Any significant dataset should be wrapped with an object. Especially so if multiple scientists/programmers are pulling data from the dataset. Ever tried multiple algorithm variants? Wrapping an algorithm in an object and then passing it to the data provisioning code means you can be 100% sure that the only thing that has changed in your code is the algorithm.
5olidor 11y ago
Does this also apply to large industrial labs, e.g. MSR, IBM Research, Google research etc.? Or are there software developers who work together with researchers in those places, perhaps resulting in better quality code?
- magicbuzz 11y ago
  I would expect that most of the folks working at computer company labs would write their own code and be very knowledgeable in a certain domain.
  - 5olidor 11y ago
    Agreed, I was just curious since some of the comments below seemed to imply that PhDs / PostDocs lack certain software development skills (although I personally haven't seen this). If so, this would either suggest that the code quality in computer company labs may be low, or they hire based on research potential AND ability to develop software
  - existencebox 11y ago
    Having worked as a SWE at both pure academia (as the engineer implementing for researcher-driven projects at a university) and in a similar position at a bigCO, there is certainly more "software chops" at the latter, but the general sentiment of the research types being far more competent at research than at code is not necessarily unfair. However, the "developer presence" in these groups is much higher, (and what is expected of even the pure research folks) so it's not quite as stark as outside industry. Still a different world from the pure SWE teams, however. The intesting thought I'd add to this is that the problems don't always manifest as "bad code". Things like measuring tradeoffs and compromises are not always as well thought out in the research teams. (I'd mete this statement slightly, to say that researchers have very different mental models for what tradeoffs to make in development from someone with a pure developer background, and this is not always bad, just evident.)
- tostitos1979 11y ago
  Microsoft Research has RSDEs ... research software design engineers. IBM Research also has a research software engineer position, with its own career progression track. These folks are generally fantastic developers who have been professional software engineers some point in their careers. Moreover, I have never personally seen a case where a research engineer was not put on a paper for being just a coder. It is problematic when a team of scientists collaborate with a product group (not research software engineers but regular developers). In the limited projects I've been involved in, we ask if any product group engineer wants to collaborate on a paper we are writing. No one is typically interested and we would at least add them to acknowledgements. I found product group developers were interested in collaborating for patent submissions. I definitely don't want to generalize but take this as a data point.
OopsCriticality 11y ago
> I wrote the majority of the analysis code for one of the chapters of the IPCC 5th Assessment and yet was told my name couldn't be put anywhere as I wasn't a 'scientist'. Can you be more specific about the nature of your contributions? Writing code would seem analogous to performing experiments, which alone is insufficient to warrant authorship (but should get an acknowledgment).
- geebee 11y ago
  You raise an important question. It is important to detail the actual contribution, since programming may be perceived as purely implementing an idea with no creative input. That said… in my career so far, I don't think I've ever seen a clean example of coding someone else's idea to spec. If you successfully take an idea from concept to working software, or if a research project requires complex software to be written, I think it is almost inevitable that the software developer will end up making a creative contribution. I'm also a bit suspicious about the justification that you can't be listed because aren't a "scientist". Of course, I don't know the details, but it seems fishy.
  - raincom 11y ago
    Isn't this simialar to what experimental physics is. Just having an idea is different from implementing and testing it.
    - geebee 11y ago
      Sorry, I really don't know anything about experimental physics, so I couldn't comment. eI see software development as somewhat similar to writing a novel - someone could give you an idea, a general outline, even a fairly detailed outline, but the writer who puts it into words will almost inevitably make a meaningful creative contribution. Here's one way to think about it - if you had two different writers, or software developers, how different would the end product look?
  - OopsCriticality 11y ago
    > You raise an important question. It is important to detail the actual contribution, since programming may be perceived as purely implementing an idea with no creative input. Beyond creative input, there's also the question of liability in case of academic misconduct. I'd question if someone acting in a technician role (say a programming professional) would have a full understanding of the manuscript and so could accept authorship; they may also lack the ability to check the findings of the manuscript which is also problematic. > If you successfully take an idea from concept to working software, or if a research project requires complex software to be written, I think it is almost inevitable that the software developer will end up making a creative contribution. I think we need to be careful about how we define creative contribution; there's a line to be drawn somewhere in there between a creative contribution for the sake of the software (i.e., speeding up execution time by 4x for a simulation running on a cluster) versus a creative contribution that enables the research (i.e., a novel real-time optimization algorithm for electrostimulation). The latter certainly warrants authorship, but I don't think the former does. > I'm also a bit suspicious about the justification that you can't be listed because aren't a "scientist". Of course, I don't know the details, but it seems fishy. Ditto, which is why I asked. My first thought was that it sounded like scholarly authorship was poorly explained and if so I was hoping to help out with that.
xiaoma 11y ago
Limited OO use sounds like a very good thing unless you're building a gui.
- jdminhbg 11y ago
  It's not "limited OO" in the sense of "well-reasoned FP instead," but rather piles of copy/pasted procedural code.
  - Retra 11y ago
    If you're not doing inheritance (which you often aren't,) a module of well-designed functions does a pretty good impression of OOP.

tensor 11y ago

Some would argue that no OO use is a good thing (assuming you are referring to object oriented programming). Expanding on this, writing research code is quite a bit different than writing production code. In particular, requirements can change quickly and drastically. This means that very open designs where you don't hide data (e.g. as is common in OO programming) are often the best designs.

wtf23737 11y ago
just no.
- camkego 11y ago
  This.
pathelectronica 11y ago
> requirements can change quickly and drastically the implication that this threat is not present for production code is completely ridiculous.
mbreese 11y ago
Research code is production code and should be treated as such. It doesn't matter if requirements change quickly, you still need to treat it like it's production code... you document, test, and deploy it. And you should include the ability to adapt the code to flexible requirements in your initial design. Just because you may only use the program once doesn't mean that you can neglect good software engineering practices. I run into bad code all of the time (some of it mine!) in our research, and it's very frustrating.
Retric 11y ago
If your research code is a big ball of mud then all it's results are meaningless. And all your research should be rejected by any competent journal. Sadly, journals tend to ignore such huge issues. But, hey it's not like reproduceable results are important in science or anything.
- mbreese 11y ago
  The more computational journals are keeping track of this. I've had a manuscript returned for revision because I didn't have enough unit testing in the code. Wet-lab journals, on the other hand, probably let a lot of stuff go on the computation and statistics than they should.
magicbuzz 11y ago
Any significant dataset should be wrapped with an object. Especially so if multiple scientists/programmers are pulling data from the dataset. Ever tried multiple algorithm variants? Wrapping an algorithm in an object and then passing it to the data provisioning code means you can be 100% sure that the only thing that has changed in your code is the algorithm.

5olidor 11y ago

Does this also apply to large industrial labs, e.g. MSR, IBM Research, Google research etc.? Or are there software developers who work together with researchers in those places, perhaps resulting in better quality code?

magicbuzz 11y ago
I would expect that most of the folks working at computer company labs would write their own code and be very knowledgeable in a certain domain.
- 5olidor 11y ago
  Agreed, I was just curious since some of the comments below seemed to imply that PhDs / PostDocs lack certain software development skills (although I personally haven't seen this). If so, this would either suggest that the code quality in computer company labs may be low, or they hire based on research potential AND ability to develop software
- existencebox 11y ago
  Having worked as a SWE at both pure academia (as the engineer implementing for researcher-driven projects at a university) and in a similar position at a bigCO, there is certainly more "software chops" at the latter, but the general sentiment of the research types being far more competent at research than at code is not necessarily unfair. However, the "developer presence" in these groups is much higher, (and what is expected of even the pure research folks) so it's not quite as stark as outside industry. Still a different world from the pure SWE teams, however. The intesting thought I'd add to this is that the problems don't always manifest as "bad code". Things like measuring tradeoffs and compromises are not always as well thought out in the research teams. (I'd mete this statement slightly, to say that researchers have very different mental models for what tradeoffs to make in development from someone with a pure developer background, and this is not always bad, just evident.)
tostitos1979 11y ago
Microsoft Research has RSDEs ... research software design engineers. IBM Research also has a research software engineer position, with its own career progression track. These folks are generally fantastic developers who have been professional software engineers some point in their careers. Moreover, I have never personally seen a case where a research engineer was not put on a paper for being just a coder. It is problematic when a team of scientists collaborate with a product group (not research software engineers but regular developers). In the limited projects I've been involved in, we ask if any product group engineer wants to collaborate on a paper we are writing. No one is typically interested and we would at least add them to acknowledgements. I found product group developers were interested in collaborating for patent submissions. I definitely don't want to generalize but take this as a data point.

OopsCriticality 11y ago

> I wrote the majority of the analysis code for one of the chapters of the IPCC 5th Assessment and yet was told my name couldn't be put anywhere as I wasn't a 'scientist'. Can you be more specific about the nature of your contributions? Writing code would seem analogous to performing experiments, which alone is insufficient to warrant authorship (but should get an acknowledgment).

geebee 11y ago
You raise an important question. It is important to detail the actual contribution, since programming may be perceived as purely implementing an idea with no creative input. That said… in my career so far, I don't think I've ever seen a clean example of coding someone else's idea to spec. If you successfully take an idea from concept to working software, or if a research project requires complex software to be written, I think it is almost inevitable that the software developer will end up making a creative contribution. I'm also a bit suspicious about the justification that you can't be listed because aren't a "scientist". Of course, I don't know the details, but it seems fishy.
- raincom 11y ago
  Isn't this simialar to what experimental physics is. Just having an idea is different from implementing and testing it.
  - geebee 11y ago
    Sorry, I really don't know anything about experimental physics, so I couldn't comment. eI see software development as somewhat similar to writing a novel - someone could give you an idea, a general outline, even a fairly detailed outline, but the writer who puts it into words will almost inevitably make a meaningful creative contribution. Here's one way to think about it - if you had two different writers, or software developers, how different would the end product look?
- OopsCriticality 11y ago
  > You raise an important question. It is important to detail the actual contribution, since programming may be perceived as purely implementing an idea with no creative input. Beyond creative input, there's also the question of liability in case of academic misconduct. I'd question if someone acting in a technician role (say a programming professional) would have a full understanding of the manuscript and so could accept authorship; they may also lack the ability to check the findings of the manuscript which is also problematic. > If you successfully take an idea from concept to working software, or if a research project requires complex software to be written, I think it is almost inevitable that the software developer will end up making a creative contribution. I think we need to be careful about how we define creative contribution; there's a line to be drawn somewhere in there between a creative contribution for the sake of the software (i.e., speeding up execution time by 4x for a simulation running on a cluster) versus a creative contribution that enables the research (i.e., a novel real-time optimization algorithm for electrostimulation). The latter certainly warrants authorship, but I don't think the former does. > I'm also a bit suspicious about the justification that you can't be listed because aren't a "scientist". Of course, I don't know the details, but it seems fishy. Ditto, which is why I asked. My first thought was that it sounded like scholarly authorship was poorly explained and if so I was hoping to help out with that.

xiaoma 11y ago

Limited OO use sounds like a very good thing unless you're building a gui.

jdminhbg 11y ago
It's not "limited OO" in the sense of "well-reasoned FP instead," but rather piles of copy/pasted procedural code.
- Retra 11y ago
  If you're not doing inheritance (which you often aren't,) a module of well-designed functions does a pretty good impression of OOP.