4 ms·

I recently quit a job as a software engineer on a research project for all the reasons mentioned. They want grunts who don't care about project management or go

I recently quit a job as a software engineer on a research project for all the reasons mentioned. They want grunts who don't care about project management or good engineering practices and they will belittle you as not worth the money even when you took a (theoretical, as I am overemployed) massive paycut to go and work for them. I made 1/3 there compared to the jobs I am interviewing for to replace it. These people desperately need software help and have worthwhile projects that need software help. They also need to learn to collaborate over just parsing out obscure goals and being upset when what they get matches their three sentence email and not their vision in their head.

bmitc 4y ago
Yea, it's frustrating because there are indeed some super cool projects just waiting to have interesting software written for them, especially in the more hardware/physical oriented R&D areas. There are some areas that take software seriously, but in general, I have found it really difficult to get scientists and research engineers excited about anything that they don't know about. I don't think its malice. I think its part of their education and institutionalization couple a little with arrogance and somewhat blind view towards areas outside their expertise. I have given presentations before, and I usually only got interesting feedback from non-PI level workers. I always tried to paint a picture that we were doing state of the art technology development, so why shouldn't state of the art software be developed to support that?
- cozzyd 4y ago
  On the flip side of that, many experiments run for decades or longer, and "state of the art" software is often a poor impedance match for that.
  - throwawaysleep 4y ago
    State of the art doesn't necessarily mean React. Just something with unit tests and more than one giant 5000 line file. If the requirement is that it lasts 30 years, you can choose more timeless languages like C, over say Rust.
    - cozzyd 4y ago
      I guess a better word for that might be "not terrible" :). And yes, way too much research code reads like somebody's first coding project.
  - bmitc 4y ago
    I'm not convinced of that. Because what I've seen (several times) on a somewhat small scale perhaps, is someone's code, sometimes written decades ago, never maintained upgraded or properly developed used for years on a system and eventually it gives out, forcing basically a rewrite. And state of the art can mean different things for differently scaled projects.
    - cozzyd 4y ago
      I've certainly had to resurrect research code written 10+ years before I got to it. It's usually not too hard as long as it's in C/C++/Fortran and doesn't have too many dependencies. Something like Matlab, on the other hand, is often a nightmare...
    - cycomanic 4y ago
      You have too look at the big picture though. How much time would have been needed to train the PhD student (might have been even the PI) to write more maintainable code 10 years ago, and maintain that code to "best practices" over the time, compared to write that it works once and need to rewrite once after 10 years? Moreover, people tend to forget that scientific software evolves much more slowly than commercial software and the number of people who work on this are much fewer, so why always adopt the newest method, if the code is still working. Show me commercial projects that have had the stability of something like LAPACK or BLAS, those essentially had a stable API for > 50 years now, with very few bugs.
      - bmitc 4y ago
        It's okay for things to start off rough, especially depending on the context. But there's often not much excuse for things to be taken seriously and evolved. But to be honest, I've seen this in industry as well from software developers. So maybe it's just a law of software development that's amplified in scientific software.
- cycomanic 4y ago
  It's very interesting that you say you find it difficult to get scientists excited about anything, because I am told pretty much the opposite. To give you the perspective, I'm a scientist and my partner is a GP. I obviously have many friends who also work in science and one thing that my partner noticed quite early after we met is that the scientist friends (and myself) would ask a lot of questions when she was talking about her work, and really tried to understand reasons, mechanisms etc.. She noted that this was very different to almost everyone else. I can definitely tell you that if I talk about my work non-scientists become quickly uninterested.
  - bmitc 4y ago
    In my experience, scientists do not like anything that they perceive as "soft". I like to say that the hard sciences are actually the easy ones and the soft sciences are actually the hard ones. (As it turns out, Herbert Simon also said this.) The point is that when you discuss things like design, complexity, architecture, systems thinking, etc., I've found that this does not stick, so to speak. Scientists do not see beauty in form equaling function. In my experience, scientists like learning about things they consider science-y enough for their tastes. This is all painting with broad strokes, but in my experience, the most interesting intellectual conversations I've had have been with people with the least amount of degrees. I always feel when I talk to someone with a Ph.D., they're afraid to let it be known someone might know more than them and if that happens, then it's suddenly something that's not important. The book Disciplined Minds gets at this some. It's part institutionalization and other things. My gut feeling is that it's exacerbated by publish or perish culture and the push to always be right. So people that are brought up through that are intellectually risk averse.
    - cycomanic 4y ago
      bmitc 1 day ago | parent | context | flag | on: Science needs more research software engineers > In my experience, scientists do not like anything that they perceive as "soft". I like to say that the hard sciences are actually the easy ones and the soft sciences are actually the hard ones. (As it turns out, Herbert Simon also said this.) The point is that when you discuss things like design, complexity, architecture, systems thinking, etc., I've found that this does not stick, so to speak. Scientists do not see beauty in form equaling function. In my experience, scientists like learning about things they consider science-y enough for their tastes. In my experience what you say is true for engineers (and software engineers are amongst the worse in this respect), but much less so for scientists from the hard sciences (physicists, chemists, biologists). You can also see that from many famous physicists being involved in philosophy, while many engineers consider philosophy "irrelevant talking about nothing". > This is all painting with broad strokes, but in my experience, the most interesting intellectual conversations I've had have been with people with the least amount of degrees. I always feel when I talk to someone with a Ph.D., they're afraid to let it be known someone might know more than them and if that happens, then it's suddenly something that's not important. Maybe we converse in very different cycles, but generally for people I know the "higher up the foodchain" the more they ask even very basic questions. In my experience you can often distinguish who is the most accomplished scientists in the room by listening who is asking the most basic questions. All this is obviously anecdotal evidence, but is reflected also by famous accounts like e.g. Feynman's account of the challenger commission.
titanomachy 4y ago
Are you saying that you hold multiple full-time jobs?

bmitc 4y ago

Yea, it's frustrating because there are indeed some super cool projects just waiting to have interesting software written for them, especially in the more hardware/physical oriented R&D areas. There are some areas that take software seriously, but in general, I have found it really difficult to get scientists and research engineers excited about anything that they don't know about. I don't think its malice. I think its part of their education and institutionalization couple a little with arrogance and somewhat blind view towards areas outside their expertise. I have given presentations before, and I usually only got interesting feedback from non-PI level workers. I always tried to paint a picture that we were doing state of the art technology development, so why shouldn't state of the art software be developed to support that?

cozzyd 4y ago
On the flip side of that, many experiments run for decades or longer, and "state of the art" software is often a poor impedance match for that.
- throwawaysleep 4y ago
  State of the art doesn't necessarily mean React. Just something with unit tests and more than one giant 5000 line file. If the requirement is that it lasts 30 years, you can choose more timeless languages like C, over say Rust.
  - cozzyd 4y ago
    I guess a better word for that might be "not terrible" :). And yes, way too much research code reads like somebody's first coding project.
- bmitc 4y ago
  I'm not convinced of that. Because what I've seen (several times) on a somewhat small scale perhaps, is someone's code, sometimes written decades ago, never maintained upgraded or properly developed used for years on a system and eventually it gives out, forcing basically a rewrite. And state of the art can mean different things for differently scaled projects.
  - cozzyd 4y ago
    I've certainly had to resurrect research code written 10+ years before I got to it. It's usually not too hard as long as it's in C/C++/Fortran and doesn't have too many dependencies. Something like Matlab, on the other hand, is often a nightmare...
  - cycomanic 4y ago
    You have too look at the big picture though. How much time would have been needed to train the PhD student (might have been even the PI) to write more maintainable code 10 years ago, and maintain that code to "best practices" over the time, compared to write that it works once and need to rewrite once after 10 years? Moreover, people tend to forget that scientific software evolves much more slowly than commercial software and the number of people who work on this are much fewer, so why always adopt the newest method, if the code is still working. Show me commercial projects that have had the stability of something like LAPACK or BLAS, those essentially had a stable API for > 50 years now, with very few bugs.
    - bmitc 4y ago
      It's okay for things to start off rough, especially depending on the context. But there's often not much excuse for things to be taken seriously and evolved. But to be honest, I've seen this in industry as well from software developers. So maybe it's just a law of software development that's amplified in scientific software.
cycomanic 4y ago
It's very interesting that you say you find it difficult to get scientists excited about anything, because I am told pretty much the opposite. To give you the perspective, I'm a scientist and my partner is a GP. I obviously have many friends who also work in science and one thing that my partner noticed quite early after we met is that the scientist friends (and myself) would ask a lot of questions when she was talking about her work, and really tried to understand reasons, mechanisms etc.. She noted that this was very different to almost everyone else. I can definitely tell you that if I talk about my work non-scientists become quickly uninterested.
- bmitc 4y ago
  In my experience, scientists do not like anything that they perceive as "soft". I like to say that the hard sciences are actually the easy ones and the soft sciences are actually the hard ones. (As it turns out, Herbert Simon also said this.) The point is that when you discuss things like design, complexity, architecture, systems thinking, etc., I've found that this does not stick, so to speak. Scientists do not see beauty in form equaling function. In my experience, scientists like learning about things they consider science-y enough for their tastes. This is all painting with broad strokes, but in my experience, the most interesting intellectual conversations I've had have been with people with the least amount of degrees. I always feel when I talk to someone with a Ph.D., they're afraid to let it be known someone might know more than them and if that happens, then it's suddenly something that's not important. The book Disciplined Minds gets at this some. It's part institutionalization and other things. My gut feeling is that it's exacerbated by publish or perish culture and the push to always be right. So people that are brought up through that are intellectually risk averse.
  - cycomanic 4y ago
    bmitc 1 day ago | parent | context | flag | on: Science needs more research software engineers > In my experience, scientists do not like anything that they perceive as "soft". I like to say that the hard sciences are actually the easy ones and the soft sciences are actually the hard ones. (As it turns out, Herbert Simon also said this.) The point is that when you discuss things like design, complexity, architecture, systems thinking, etc., I've found that this does not stick, so to speak. Scientists do not see beauty in form equaling function. In my experience, scientists like learning about things they consider science-y enough for their tastes. In my experience what you say is true for engineers (and software engineers are amongst the worse in this respect), but much less so for scientists from the hard sciences (physicists, chemists, biologists). You can also see that from many famous physicists being involved in philosophy, while many engineers consider philosophy "irrelevant talking about nothing". > This is all painting with broad strokes, but in my experience, the most interesting intellectual conversations I've had have been with people with the least amount of degrees. I always feel when I talk to someone with a Ph.D., they're afraid to let it be known someone might know more than them and if that happens, then it's suddenly something that's not important. Maybe we converse in very different cycles, but generally for people I know the "higher up the foodchain" the more they ask even very basic questions. In my experience you can often distinguish who is the most accomplished scientists in the room by listening who is asking the most basic questions. All this is obviously anecdotal evidence, but is reflected also by famous accounts like e.g. Feynman's account of the challenger commission.

titanomachy 4y ago

Are you saying that you hold multiple full-time jobs?