24 ms·

Science needs more research software engineers

• grillorafael 4y ago
  I would happily stop everything that I'm doing to become an RSE if it paid well enough. I really like the field and what it is but if it doesn't pay the bill then it doesn't pay the bill.
• ethanwillis 4y ago
  Well, then science needs to do more to create an environment where software engineers would want to be an RSE. From my experience "science" does not appreciate RSEs, the compensation is bad, the freedom is bad, and you get all of the big company bureaucracy from the University. It's just a bad environment and to top it off, even if you have a PhD many will see you as "less than."
  • cameldrv 4y ago
    IME that's all true. To say that they need to do more to create a more positive environment for research engineers though depends on the perspective. From the perspective of the university and the PI, they can just get a graduate student to do the work for 1/5 the cost.
    • BeetleB 4y ago
      > From the perspective of the university and the PI, they can just get a graduate student to do the work for 1/5 the cost. That is what they do. However, some projects grow in scale where a grad student cannot handle it well, or if they try, it will be a detriment to their research. To give you an example, only one of the PhD students (non-CS engineering) in my group and the ones around me had taken data structures or algorithms - yet everyone's thesis was "computational" (i.e. numerical computation, etc). None of the advisors would appreciate their students going off and taking serious CS courses. When a research team wants to go to the next level, they need to hire someone with better SW skills.
  • de6u99er 4y ago
    Apart from compensation and freedom you also get micro managed by someone who might know about the science part, but is really bad on the software architecture and engineering part.
  • mywaifuismeta 4y ago
    The last point is important. In my experience it's mostly about status. RSEs are always seen as inferior to researchers (research scientists) who supposedly come up with the "big ideas" while RSEs merely implement the stuff they're told to do. In reality the line is much blurrier. There can be no innovation and iteration without implementation of ideas and the RSE work is just as important. But unless this view changes, nobody wants to be an RSE.
    • JetAlone 4y ago
      According to a quick search for the case of researchers, and the stereotype of most staff in academia across the board being underpaid, this means academic researchers would have even less incentives/perks/reward, because now they have one less beneficial status differential. Is there some way to make human respect feel like it's not a zero sum game? The world may never know.
      • barry-cotter 4y ago
        > Is there some way to make human respect feel like it's not a zero sum game? Respect is non zero sum but status is definitionally a positional good. If you’re number 1 someone else isn’t. There can be uncertainty about status but ambiguity always collapses eventually. Everyone can be treated with respect but there will be a prestige or dominance hierarchy in any group of humans, subtle as it may be.
    • amelius 4y ago
      Yes, I think RSE is often viewed as a laboratory assistant.
      • auxym 4y ago
        Not just viewed, but literally called an assistant in the official title. I am a licensed professional engineer (mechanical) and I work in academia (though my work is varied and does involve some code), my official title (in french) is "Professionnel de recherche", which translates to Research Professional. It is my understanding that in most of the english-speaking world, this position (someone that works for a research lab, who is qualified above the technician level, but not a PI nor a postdoc or student) is called "Research Assistant". I write "Research Engineer" on my résumé/linkedin, because TBH for most people who don't have much experience in academia, "Research Assistant" sounds like an admin assistant (secretary).
  • tonto 4y ago
    your premise is that it's not being sold right but maybe you just need more people to give their perspective. I might be considered a research software engineer in the US. In my job, I get paid (well) to develop open source software for data visualization, and I work remotely. I get to maintain existing tools and develop new ones. there is an interesting global community that asks questions daily, and I am constantly learning. I'm not working under the thumb of overbearing people, just delivering results and contributing to new grant goals to continue the funding. these positions are not necessarily common, but are interesting and cool.
    • FlyingRobot 4y ago
      I'll offer another positive perspective. I've worked as a software engineer within the Department of Energy's National Laboratory System for 15 years, and I really enjoy it. Software is a major element of much of the laboratory's work, and in some cases such as mine software is the main product. We enjoy autonomy, lead projects as PIs, and develop mostly open source software. We are also hiring https://nrel.wd5.myworkdayjobs.com/NREL/2/refreshFacet/318c8bb6f553100021d223d9780d30be https://nrel.wd5.myworkdayjobs.com/NREL/2/refreshFacet/318c8....
      • artwr 4y ago
        I will second this. The national labs are definitely some of the places who know what to do with research software engineers and treat them right for the most part. The Computer Systems Engineers and Software Engineers I had the chance to meet at LBNL had a decent amount of autonomy and were very good.
      • tsj5 4y ago
        Hi FlyingRobot --- I'd be interested in hearing more about your career path (and openings at NREL). What I've personally heard and experienced aligns with the overwhelmingly negative accounts elsewhere in the thread, which (I feel) have also correctly identified the misalignment of incentives that's responsible. On the other hand, I've never worked in the national lab system. If you're still monitoring this thread, I'd appreciate it if you contacted me at the email in my profile!
  • ehnto 4y ago
    Absolutely, but I will give them the hand that achieving a good software dev experience is extremely hard inside a uni. There is a number of ways that they need to match the industry and are having a tough time doing so due to the institutionalization of all the adjacent processes.
  • travisgriggs 4y ago
    This last point, and replies to this all seem to talk about the research industry. But it has been my experience that this same dynamic of being “less than” exists in any industry that is not primarily software. I am a mechanical engineer by degree, but a software engineer by 30 years of practice. That has allowed me to thrice straddle the divide where I worked on software that was not just software for softwares sake, but rather as a value add addendum/enablement to an institution whose roots were in more real-worldly devices. I also spent time on the other side of the divide working twice for companies whose primary product was software. This “less than” is a very real thing. Software is still a very new thing relatively and it’s taken over the world in just 30 years time. The power dynamics, entrenched in generations haven’t had time to rebalance. I’ve seen this run the other way as well. Where companies that are primarily software, look down on the other disciplines that participate, a necessary evil, but resented. I have yet to find a company/institution where the trifecta of mechanical, electrical, and software is balanced in mutual respect. If one such exists and you are looking for someone that would love to work in that environment, drop me a line. I’m skeptical that there are any. It seems one discipline or the other always triumphs and tramples the others down. On off axis variant of Pournelle’s iron law of bureaucracy .
    • NoahKAndrews 4y ago
      I sent you an email too, but my employer REV Robotics definitely fits this description in my opinion, especially in regards to the software team's collaboration with the electrical team.
    • rytill 4y ago
      Just curious, have you sampled many robotics companies? I have worked with dozens and it seems like there are some with well-balanced power dynamics between mechanical-electrical-software.
    • deleted 4y ago
      [deleted]
    • fburnaby 4y ago
      I believe this has less to do with software in particular and just generally reflects the typical dynamics of the line/staff distinction. https://en.m.wikipedia.org/wiki/Staff_and_line https://en.m.wikipedia.org/wiki/Staff_and_line
  • ElevenPhonons 4y ago
    That was my experience as well. The RSE role doesn't seem to fit into their current model. It doesn't work from a career path, nor from a competitive compensation standpoint. I suspect that the housing crisis is also going to push more people out of these positions. If you have a PhD in material science, physics, chemistry, biology, etc... and are reasonably knowledgable in Python (or similar), perhaps spend a year as a postdoc. After that year, seriously consider moving to a tech company.
  • epicureanideal 4y ago
    If RSEs are actually valuable but under-compensated, can someone please disrupt this industry? Would love to see more startup medicine development companies ... Alternatively, would it make sense to provide RSE services as a company, for very high hourly rates, separately from the rest of the university, so status and rules are less of a problem?
    • kyawzazaw 4y ago
      There are a few like these: 1. Benchling 2. Enable Medicine 3. Radix.bio
    • mike_hearn 4y ago
      The reason they're under-compensated is that they aren't valued, unfortunately. A lot of scientists are self taught programmers and don't know what they don't know. The other issue is that if you say you should spend half your departments budget on hiring much better paid software developers, it looks really bad: a. You're implying your colleagues are incompetent. b. You can't use that money for "science". c. It puts in painful relief that the software guys are much better paid than you are, implying that their work is more valuable than yours. But a lot of scientists put up with low pay because they believe their work is really valuable.
  • Azrael3000 4y ago
    There is simply no incentive for science to do proper software engineering. It does not directly produce papers or money, so nobody does it. I've seen it several times in my own career in several countries. PhDs having their own version of the code, incompatible with the one of the Postdoc sitting opposite of them. Each slaving away at their niche project but nobody there to bring it all together. This particular university had really great code and they thought they could sell it. But they did not even use git and when I gave a presentation there about it, I was met with absolute rejection. They don't have an idea on how much effort it takes to maintain a commercial code, unit tests, customer support etc. I'm now working at a company that produces simulation software spun out from university. Fantastic job and we do all of those things mentioned above. But obviously my paper output has been near zero even though we sometimes do cutting edge research. I wish I had a solution to the problem as so much grant money is wasted that produces a paper or two and the corresponding software for that just goes to digital nirvana, it's a real shame.
    • w0de0 4y ago
      Is there any solution to be found through analogy with established fields' relationship between engineering and research disciplines? How, for instance, do physicists share formal models which rocket engineers might apply? Certainly the physicists aren't out developing unit tests for engines which apply their theory, but nor are the engineers exasperated by physicists rejecting their standardized tools (I imagine?).
      • Azrael3000 4y ago
        Well from experience the interface is conferences and papers. On conferences you get to know the latest stuff that is going on and the implementation details can be found in papers. What needs to be said though is that reimplementing something from a paper is near impossible. I had to do that a couple of times and I was only fully successful if the paper was accompanied by some open source code as there always are tiny little edge cases or initialization details that won't be covered in the paper.
    • joshvm 4y ago
  • kimixa 4y ago
    I certainly agree - it was about 10 years ago now, but I did a physics degree and ended up pivoting to computer science. I offered to help a couple of friends in the department with their analysis, and ended up writing the majority of their code in actually analysing their experimental results. In both cases, their supervisor decided not to put my name in the resulting papers, and treated me less than a lab tech. Why would anyone want to do that, when a six-figure tech company salary is just next door?
  • jltsiren 4y ago
    While the academia has a lot of issues, software engineers should also adjust their expectations. First, if you choose to work for a non-profit, you should expect a huge pay cut. That's the way the society works. Or actually it's the opposite. If you work in a for-profit role, the society allows your employer to keep most of the value you create. If you create a lot of value, that gives you the leverage to negotiate much higher compensation than your peers in less profitable roles. Second, like in any business, you should focus on what brings the money in if you want to advance your career. You should be the PI instead of the lab technician. Instead of building the software other people ask, you should become an expert in the field and build the software other people will need in the future. For example, bioinformatics as a whole used to be more of a support role, but today there are many high-profile bioinformatics PIs. Third, many people in the industry fail to realize that you don't really work for your employer in the academia. You work to build a reputation for yourself, which makes you valuable to the employer. If you want to continue working for other people, the upward career path from support roles goes to the administration.
    • lliamander 4y ago
      > Third, many people in the industry fail to realize that you don't really work for your employer in the academia. You work to build a reputation for yourself, which makes you valuable to the employer. That feels like how it works outside of academia as well.
      • jltsiren 4y ago
        There is a huge difference: Universities don't do reseach, and they don't care about the research their employees do. If you switch jobs, your old employer doesn't hire anyone to continue your research. Instead, you usually take your project to your new job. In the industry, your employer tends to own the projects you work on.
    • arvinsim 4y ago
      My apologies for not being familiar with acronyms, but what is a PI?
  • maxwell86 4y ago
    > even if you have a PhD many will see you as "less than." Inside academia and outside of it. Inside academia you are not on a tenure track or similar, and will have to put up with a lot. Outside academia, your peers will be making 3x or more than you, working less hours, with less stress, etc. The reason RSE's jobs are hard to fill and often aren't even opened is that they don't make sense. If you are good enough for an RSE job, you will be good enough for research postions at FAANG. Those pay 10x more, so you also need someone willing to not accept that 10x pay, and also willing to work double the hours. RSEs making a reasonable pay for the skills they require make no sense either, because that would put your pay at 2x that of professors, etc.
    • robbomacrae 4y ago
      I'm curious which company pays a RS 10x more than a SE/RSE. I've found its usually SE/RSE's that make a bit more than RS's but I've never seen an RSE comp significantly outweigh an SE unless its in ML/Crypto. On that note I wish levels.fyi had RS and RSE roles...
      • epgui 4y ago
        10x is possible, but maybe 3-5x is more realistic (even for non-FAANG).
      • adw 4y ago
        10x more than academia. I’m a PhD in mineral physics turned Big Four MLE via startups and I earn about 10x what I would be on if I had stayed on the academic track.
      • exdsq 4y ago
        This isn't totally crazy but it's only if you stretch the facts enough. An RSE at Oxford will earn 32k GBP, like a postdoc. In theory, that person could be so good they could get the highest starting salary possible at a Tier 1 paying company like a research engineer at Hudson River Trading, which can be > 320k TC. So it's possible but only for a very very small number of people. 3x-5x is much more likely.
• cshenton 4y ago
  Would love to work at a University as a Research Engineer. Would not love to do it for less than I made as an undergrad out of college. Universities can avoid not paying market, but they pay more like 40-50% of market.
• spicyramen_ 4y ago
  In main FAANG labs this is already a constant Brain, Facebook Research, MSR and AWS. Researchers work with SWEs to deploy models and hire hybrids.
• powerslacker 4y ago
  If you need them then pay them.
  • Mountain_Skies 4y ago
    Pay them and treat them well and with respect. Money can only hold someone for so long if they're treated poorly and like an inferior.
• toma_caliente 4y ago
  It's already been said several times over but I'm going to throw it out anyway. I'd love to work at a university as a research software engineer. I find things like bioinformatics fascinating and even interviewed at a research university on the east coast. But, you will get paid SIGNIFICANTLY less than you would doing almost anything else, have zero autonomy, and get no respect. Those 3 things are basically what most people look for in a job.
  • philomath_mn 4y ago
    > Those 3 thing are basically what most people look for in a job. When you put it that way, it makes it so clear why nobody wants to sign up (BTW I'm the same way, would love to work on research, especially if it is something which could improve people's lives, but I'd have a hard time switching for these reasons)
  • Gatsky 4y ago
    The pay and autonomy will never improve, I can tell you that now. That’s the reality of the current system.
    • cjbgkagh 4y ago
      I’m certain the respect will not improve either. There may be steps made to improve it but they won’t stick.
  • david_l_lin 4y ago
    >you will get paid SIGNIFICANTLY less than you would doing almost anything else Definitely true. >have zero autonomy Might be true? But in the right research group, this is definitely not true. I actually find that I had far more autonomy in academia than in industry. There are far fewer deadlines, less time pressure, and more curiosity-driven projects that are entirely owned by you. >and get no respect Might be true, but I find a lot of people in industry feel they get no respect as well. As an academic, I actually found that I respected academics more than I did people in industry. Though this is entirely anecdotal. I was an academic until my 30s before going to industry, and actually find that the autonomy is the number one reason people like academia. YOU drive the projects because only YOU know the research that deeply. YOU get to be at the forefront, and YOU know things nobody else might ever know, until you get to share them through publications and presentations. Academic projects have essentially no deadline (projects are on multi-year long timescales rather than quarters or months in industry).
• justahuman74 4y ago
  Pay money.
• prpl 4y ago
  I’d been active-ish in US-RSE and worked at a national lab for 10+ years. I left about a half year ago for a position at a FANG-ish company. My pay was almost doubled (after counting bonus) and with RSUs it’s double+. I could have squeezed out another $XXk maybe when I left the national lab system, should I had stayed. The biggest reasons I left were as follows: 1. Too much legacy responsibility. Legacy responsibility is usually underfunded or unfunded but “important”. It’s a lot of work to keep the lights on. NASA usually handles this with contractors which are usually paid even less than a National Lab person. 2. Bifurcation at the lab towards very large projects or very small projects. Not a lot of stuff in the $250MM to $750MM which is large enough to have an effective team and go full time, but small enough to not be anonymous/matrixed out to subprojects. The high end is a very funny place 3. Related to (2) is not enough new blood or turnover. This may be a good thing but you can be hired as a junior engineer and end up as a senior engineer without anybody below you for years. Getting an intern is a dark art if your not a staff scientist 4. Institutional BS. Looking at you, security drones with professional certificates running qualys/nessus scans. and you too, data center people afraid of the cloud. Some to zero agility in terms of compute - everything is a nail and slurm is your hammer The university-based RSEs complain a lot about not being respected/publishing. I never really had that issue much specifically, but that may be because there’s a stronger understanding of value in the national lab system. On the other hand, poor leadership/project management means you may still be at the whim of an idiot, servicing their desire for some crappy web app project. In those cases, it’s best to ignore. I will probably go back after some time when I’m throughly annoyed by BigCorp. In the mean time, it’s fine on the other side and at least I don’t have to patch Jenkins every two weeks.
  • deleted 4y ago
    [deleted]
• PainfullyNormal 4y ago
  Where do you find these jobs? And are they only available at Universities? I'd be happy to take the pay cut to work on something that contributes more to society than getting people to click on ads, but not at a college or university.
  • ergl 4y ago
    You might be able to find research institutes or national labs (I'm a research engineer at one of these). If you're in Europe, something like the Max Planck Institute might fit the bill. Aside, can I ask why you don't want to work at a college or university?
    • PainfullyNormal 4y ago
  • mattsouth 4y ago
    in the uk, https://www.jobs.ac.uk/search/?keywords=RSE&location= https://www.jobs.ac.uk/search/?keywords=RSE&location= gets you five hits today and I expect you would find more similar posts that werent currently branded research software engineer if you looked - the term is still quite new for many academic depts. One thing you'll need is patience if you want one of these positions - I remember being offerred an academic post three months after interviewing for the position and when I queried how on earth it could have taken so long to offer the post I was met with incredulity that there was an issue at all.
  • milliams 4y ago
    We (the Society of Research Software Engineering) maintain a jobs board at https://society-rse.org/careers/vacancies/ https://society-rse.org/careers/vacancies/. Or if you are in the US, they have their own (https://us-rse.org/jobs/ https://us-rse.org/jobs/)
    • PainfullyNormal 4y ago
• deleted 4y ago
  [deleted]
• dbcooper 4y ago
  Science needs more "technicians" of all types, and less PhD students. Good luck making that transformation though.
  • zmgsabst 4y ago
    Could you elaborate on what that would look like?
    • l33t2328 4y ago
      I think they’re describing professional grad students.
  • raister 4y ago
    > Less PhD students PhD students are the backbone of any research project. They interact with postdocs and with undergrads in the research ecossystem. If anything, they will be required more in the years to come, not less.
    • dguest 4y ago
      It depends on the amount of technical debt in the project. When there's no technical debt, sure, grad students are the backbone. They can happily go about their projects and supervise undergrads, while the postdocs supervise them and apply for jobs. In a place where there's a lot of technical debt, a postdoc is going to be far more productive than a grad student. In some experiments many grad students spend years floundering around before they learn enough to cut through the cruft and do real research. It's pretty silly to fire all the postdocs just when they've managed to understand everything and replace them with new grad students. The tricky thing is keeping enough of the technical staff around, and coming up with a system where they are rewarded for eliminating debt, rather than penalized for making their own hard-earned skills obsolete.
  • Mountain_Skies 4y ago
    SOX for the head of research. There are minimal consequences for producing poor quality software that creates erroneous results and often quite a few incentives for producing inaccurate results that can be blamed on a non-human entity (the software). Even SOX with an escape clause for "good faith effort" would be a game changer since what goes on now would never be considered a good faith effort by any court. SOX hasn't been anywhere near as successful at modifying behavior as it should have been but it still has caused a good bit of change and those at the top setting priorities and incentives for those below them to align with now know there's a possibility of some hefty consequences for reckless or negligent behavior.
• Nursie 4y ago
  The other side of "We need software engineers in research" is surely the botched covid model from Imperial College way back at the beginning of the pandemic. The group that produced it had the audacity to complain that the software industry didn't make C++ foolproof enough, and it was our fault that the modelling was wrong. This shows the attitude that some folks in these places have - what you do is easy, and rather than your industry being exceptional as a profession that makes its tools open and shares knowledge openly, you are at fault for releasing things which I can get wrong. "Scientists" in this view are smart people, and anything you lesser mortals do should be obvious to them immediately, or it must be your fault. Training and experience be damned. I'd hate to work anywhere where that attitude was prevalent, or even really existed at all. And yes I realise that non-software folks often hate to work with us because of the exact same thing!
  • andi999 4y ago
    Any reference for that incident, i havent heard of it and am curious.
    • Mountain_Skies 4y ago
      Good post on the subject. https://news.ycombinator.com/item?id=30146245 https://news.ycombinator.com/item?id=30146245
    • Nursie 4y ago
      Here's the open letter criticising our entire profession for producing something as terrible as C++ and allowing scientists to get into trouble with it - http://blog.khinsen.net/posts/2020/05/18/an-open-letter-to-software-engineers-criticizing-neil-ferguson-s-epidemics-simulation-code/ http://blog.khinsen.net/posts/2020/05/18/an-open-letter-to-s... There are links back to other parts of the story in the text there. Effectively, they built a model that predicted some real worst-case stuff, and used that as the basis for advice they sent out to try and affect government policy in a bunch of places. When the code was finally released and examined, it was in a bad state - 15k lines in one source file, race conditions, single name globals reused all over the place. The results were trash. Then the blame started to fly.
• xroja 4y ago
  I've worked at a top research university for the past 15 years as a research software engineer. Many of these comments have some truth but a lot are definitely not universally true. Poor compensation: Yup, I'll cop to that. I could be making way more in "industry" but I make enough to live and I don't build ad-ware and do take pride in my work. Being a PhD but little respect: I actually have the opposite experience. I'm not a PhD, I don't even have a Masters. But I can write decent software and I get respect and recognition from my PIs and other top-tier faculty that we've collaborated with. I feel like I've gotten just as much respect with my humble BS in Comp Sci from a state school than if I got a MS/PhD. Micromanagement: I have the total opposite experience. I've worked with the same group for a long time, maybe that changes things. We apply for and receive grants to work on projects. We make high-level assertions for what we will do during the course of the grant and our PI gives us lots of lee-way to meet the goals of the grant in whatever technical way gets results, is FAIR, and open. Granted, I may have just lucked out here.
  • breck 4y ago
    I also did a couple years at a university as a research software engineer and my experience matches yours.
  • titanomachy 4y ago
    I also had a good experience as a research software engineer. Tons of autonomy and fun work. Reasonable expectations and no overwork. But it was barely a job, I got paid like a tenth of what I make in industry. Once I saw the kinds of offers friends were getting, I was out. (My research institute didn’t have a concept of “research software engineer” so I was paid as a generic research assistant.)
  • prpl 4y ago
    I kind of alluded to this, but my experience was slightly similar. B.S. in Physics (w/ years of significant undergrad research), but worked with PhDs+ in national lab scenario. At some point everybody assumes you have an PhD. Little to no micromanagement. It was more professional scenario. IMO an RSE at a mid-ranked state school hired by a PI and embedded into a research group, compared to an RSE at a top-tier research University - at a lab or institute that employs many engineers, they will have vastly different experiences and probably pay as well. There's exceptions to the rule, but this is generally my experience. There's a good chunk of both jobs, but I would not work the former.
• bmitc 4y ago
  I don't think scientists want research software engineers. PIs typically like to assume any and every role on a project, including software design. They don't like handing things off and letting people build modern, best practices software. You start talking about source code control, package managers, a language used in industry that they've never even heard of and don't care enough to learn about, etc., and you just get asked "why can't you just write it in Python or C because those are the only two things I'm remotely familiar with and zip it up and put it on a USB drive?". Heaven forbid you actually want to do something novel, no matter how enabling it could be. I have been continually inspired by Bret Victor's Seeing Spaces talk and have always wanted to build such a system described for a lab, but unfortunately, it was really hard to get anyone interested much less excited by such an idea. Scientists are far more institutionalized, conservative, and myopic than they admit. Physicists don't even use the proper word for code. They call it "codes". (I've seen in first-hand, in research papers, and in published books. It's endemic.) I've literally sat in meetings trying to get access to scientific projects I'm interested in and applied for jobs even at the university I already worked at in similar capacities, and yet, it's filled with PIs just wanting grunt labor (i.e., what they're used to from graduate students) to build things that they understand or you absolutely never hear back or the pay is a third of what you already make. And then there's pockets of software people in this niche world that only want to do things their way, because that's all they've ever done, and they can really be quite obstinate. They're like this massive pillar you must somehow work around. I really, really, really like writing software in scientific contexts, but it's such a quagmire. I'd like to return to that world some day with the lessons I've learned outside of it. Software in support of an interesting domain or context is just really fun. I've also tried getting a job at an architecture firm that had some very neat ideas for building out a tool, but the response was basically the same as described above.
  • throwawaysleep 4y ago
    I recently quit a job as a software engineer on a research project for all the reasons mentioned. They want grunts who don't care about project management or good engineering practices and they will belittle you as not worth the money even when you took a (theoretical, as I am overemployed) massive paycut to go and work for them. I made 1/3 there compared to the jobs I am interviewing for to replace it. These people desperately need software help and have worthwhile projects that need software help. They also need to learn to collaborate over just parsing out obscure goals and being upset when what they get matches their three sentence email and not their vision in their head.
• jknoepfler 4y ago
  I haven't seen a single competetive job posting from a research institution in 10 years of software development, despite having the desire and acumen to seek them out. I don't expect that to change. If I care enough about a thorny research problem, I'll start my own company, do my own research, and maybe hire some academics as bargain-barrel contributors to do grunt analytics work... the last thing I'll do is join a research mill as a cost-center tech hire. "Most RSEs have a PhD"... talk about selecting aggressively for people with no experience building functional software. Hiring only PhDs to write research software smacks of hiring only white males who can drink and play golf to function as business executives. You're selecting for ability to posture, not deliver value.
  • mikebenfield 4y ago
    Getting a PhD (ie, passing an apprenticeship in how to do research in an academic setting) is a lot more strongly related to being a RSE than playing golf is to running a business.
• denkmoon 4y ago
  I was a "research software engineer" for a major research organisation in Australia. There just weren't the resources to achieve what was desired. It would have taken me at least 24 months to modify the existing ecological modelling framework to work in the way they were asking for, they wanted it done yesterday, and there was no money to bring on another engineer to spread the load. Instead I got given an intern from the internship program. While I tried my best to give them a good experience, let's just say that wasn't the boon to productivity my manager thought it was going to be. To make matters worse, one of the researchers in the PhD program had been told these changes would be ready shortly before I even started, and the PhD candidate had made a series of choices about their studies relying on those changes being available. Felt bad to have to break the bad news to him after the lab manager had been blowing smoke up his ass for so long.
• itsmeacupoftea 4y ago
  I can only speak for how it appears in my country (Australia), but it seems like university academic staffing models in general are broken. I have 15+ years experience in service providers in Cybersecurity and SE roles. I have 2 Masters degrees with GPA7/HD averages, and love to teach. There is nothing I'd love more than to teach university level topics with a primary teaching focus, and research as a secondary. It's not about the money - they can pay me less than the Australian average wage and I'll be happy. Over the last 2 years I've interviewed at multiple universities for Level A (Associate Lecturer) and they all had similar requirements that I've summarized below: > you must have completed a PHD. > You must propose and perform original research, assist in upper level academics research, and publish X times per year. > you must participate in fundraising, complete your own grant applications and assist upper level academics in their fundraising & applications. > you must supervise and advise X number of postgraduate students thesis/dissertations. > you must teach both undergraduate and graduate level classes across 3 trimesters. > the position is fixed term for 12 months, rather than ongoing/permanent, with no job security and you are required to reapply each year. Most current academics I know from Level A to Level E, say they don't particularly want to teach, and just wish they could focus on their research and fundraising to support their research. I met other applicants at networking and alumni events like myself who would like a teaching focus, rather than a research focus. There's a supply of "specialists", but an insistence on "generalists". As the fixed term aspect tends to apply to research fellows as well, the risk from the lack of security combined with the extremely low pay makes it very difficult to enter, and remain, in academia. I can afford to live on $65k a year, but the possibility that every year I may be faced with periods of unemployment and the requirement to relocate thousands of km away makes it a challenging prospect.
  • aragilar 4y ago
    It comes down to teaching not bringing in money—enrolments (either domestic via the government, or paying international students) and grants bring in money (those requirements are basically "can bring in grants/other funding"). Even with ongoing positions, things like COVID (or even a change of focus) can cause the positions to end (and the ability to bring in grants would play a key roll in who stays and who goes). Any substantial change in the model is a matter for government, not the unis.
• User23 4y ago
  The easy fix is to move software devs into the administration department and pay them administrator salaries.
• throwmamatrain 4y ago
  This looks great, and I would have loved to see this when I was in the lab. When the software industry says to you: "We will nearly triple your salary, you don't have to work weekends, and you also don't have to feed the mice on a Sunday night." You will 100% take this deal. I was a 10yr+ academic tool maker in biochemistry, built cutting edge microscopes, hardware, and image analysis software. My lab was successful in our field. I got some papers out of the deal. I also saw things that no human had seen before in a microscope. I worked with very interesting people from around the world. The work in academia is great. You're moving the needle, new data, new failures. These are the perks. It is also highly possible that you have complete creative control of your project. I did, and it was amazing. Custom designed UIs to streamline our experiments, derived from watching students use the system to do their work. A decathalon of software design. Some reality: Your PI and organization will never compensate you the way the software industry will. In pay, expectations, and benefits. When you're over 30, and you don't technically have a real 401k, you are still paying your student loans that you needed to get into this field, and you're still in the same shitty apartment, something has to give. Comparison is the thief of joy, and when you see your cohort of computer science graduates your joy will be stolen :). It's good, honest work. A short tour of duty would be useful, and can teach you the difference between splitting the atom and splitting the check. Academia, at least in bioscience, is still very much an ivory tower. You don't have enough letters after your name to matter, and you will likely be a pet instead of a peer. Don't stay underwater for too long. Life is short. :D
  • frozenport 4y ago
    What's Nico Stuurman up to these days? https://valelab.ucsf.edu/nico_stuurman/ https://valelab.ucsf.edu/nico_stuurman/
    • throwmamatrain 4y ago
      Ah, the uManager guys! Great software, sits on top of ImageJ (open source image analysis). Our software was a custom C+Win32 app that was ported from CodeWarrior on MacOS 7/8. Windows timers were so crap that I ended up using Ryan Geiss' timer from Milkdrop: http://www.geisswerks.com/ryan/FAQS/timing.html http://www.geisswerks.com/ryan/FAQS/timing.html Yes, that Ryan Geiss, the Winamp one. He now works for Nvidia I think. Our machines were bristling with serial ports, driving high speed filter wheels, lasers of every color we could get our hands on, special shutters coated with teflon, fast expensive cameras, and more! Their work is very much in my old field, I was in bioenergetics, specifically mitochondria and their dysfunction. Thanks for the link down memory lane!
• lifeisstillgood 4y ago
  There is (slowly) becoming a concept of a "tour of duty" from industry into working in dot-gov (like UK digital service). Is there room for something similar in academia - to bolster the profile of the RSE idea, to get some support funding etc?
• Existenceblinks 4y ago
  Majority of frontend developers don't even appreciate a simple aspect of science; browser performance get worse as javascript bundle size increases. I think we also need software engineers to appreciate more on science.
• analog31 4y ago
  I'm a R&D scientist, working in industry. My work site has a full blown software department. Yet I do all of my own coding. Pay and status are not issues -- if I had a programmer working for me, they'd get paid according to industry standards. Some other issues include: 1. Knowing that software development can be a black hole. Nearly half a century after The Mythical Man Month, management of software projects remains an unsolved problem. And we're not gifted managers to begin with. When I do it myself, if nothing else, I have a pretty good sense of when it will be done. 2. What to do if there's not enough programming work for a full time programmer, and that person doesn't want to work on other things. 3. It seems like programmers with domain skills related to R&D, such as math, are in particular demand. 4. It takes a certain temperament to hang in an R&D setting. A lot of engineers just hate it: The rapidly shifting requirements, and the knowledge that something they make will only be used once, or even wrecked. The dirty secret is that the same programming work that's a step down for a commercial developer, might be a step up for a grad student or researcher who wants to learn a marketable skill. Also, a multi-disciplinary team, including people who can program as needed but also do other things, can be quite agile. My team has a software engineer, but he doesn't do very much coding for us. He has actual R&D projects of his own. Yet he can help us at a higher level, for instance giving guidance on how we can write better code, and make better use of tools. This brings us a lot of benefit without wasting his brain cells on mundane coding tasks.
• seydor 4y ago
  Most academic code is crap so there is that. the coworkers will be very bad programmers, difficult to advance (you should always work with people better than you). Computational papers are not often appreciated. Too much talk and too many pivots , too little code, at least too little pleasant code-writing. Unless you have a lot of ego, you will be ignored. I am sure a lot of academic code would be written by the open source community, but alas , researchers are very stingy giving out their data
• robbomacrae 4y ago
  This hit pretty close to home for me, as I started as a programming oriented PhD and then transitioned into a SV full software engineer over 10 years. I am exactly like they mentioned "Many RSEs started out as PhD students or postdocs who worked on software to support their own project. They realized that they enjoyed that part of the job more than the actual research." I think the advice in the interview is pretty sound. OSS is a great way to contribute and become in demand. But the main issue here is that there are not enough RSE's. "If RSEs can get the recognition and rewards that they deserve, then the career path will be that much more visible and attractive." Here are my two cents: 1) Academia will never compete in salary with big tech (except maybe in the big Ivy League schools). Universities should promote spin offs, help with patents, run regular hackathons and an incubator program. That will give engineers an incentive to at least start their career in an academic lab. 2) Some of the best science happens in startups and org x labs, not just academia. But even here I found there are very few companies that value an RSE. You're either an RS, or an SE and the extra R or E is in title only. They will see you as a researcher who knows a bit of python, or as an engineer who wasted years in graduate school. You will be interviewed as one or the other. You will be paid as one or the other. I've never seen a company appreciate the value of fully grokking or contributing to the researchers algorithms and Jupiter notebooks and then architecting an efficient implementation at scale. You will also be kept in your swim lane. Researcher? Can't touch the production systems. Engineer? Good luck getting a patent application going. If there is a hiring manager out there reading this... please consider making a true RSE interview and role.
• j7ake 4y ago
  Research software engineers should be PIs themselves so they can choose what they want to work on, who to work with, and hire trainees to mentor the next generation of research software engineers. RSEs are a different breed from SEs from industry, the work is not about scaling to a billion people but being nimble and adaptive to new results and ideas. If one wants to see change, then hire senior PI positions who do research software engineering. This will help create a hub and prevent the feeling of being a “pet” in an experimental lab.
• deleted 4y ago
  [deleted]
• Mandatum 4y ago
  Science can pay me more.
• pojzon 4y ago
  In the world of ever increasing cost of living, sane ppl will always take best compensation offers. Everyone has the same thinking that we will be here 40-70years and our finite time is worth more and more. Im sorry for the ppl tho dont have options to do better for w/e reasons.
• ThePhysicist 4y ago
  The first thing I did for my PhD in experimental quantum computing was to rewrite the lab control equipment in Python. It was written in LabView and while it worked well it was an absolute mess of (literally) tangled spaghetti code. I designed and implemented one of the first MVC frameworks for quantum computing in Python. It had an interactive IDE where you could execute scripts and store snippets (e.g. for qubit calibration), an instrument manager that allowed initializing and managing all lab equipment (locally and over the network), a data manager that allowed capturing and exploring measurement data in real-time and instrument frontpanels that allowed interacting with instruments (e.g. microwave sources or waveform generators). Initially my supervisors were very skeptical and thought I'd waste my time, as they did not see any value in software development. Having a programmable and well organized software framework was absolutely instrumental for my work though, and I think I couldn't have succeeded at my experiments (demonstrating quantum speed-up for a simple quantum algorithm) without it. Today the software is still in use in several labs and as far as I know they also hired a research software engineer to keep developing it.
  • nurbl 4y ago
    This sounds like a good outcome. From my experience, the usual reason small labs setups use things like LabVIEW is that they are usually built by one PhD student without experience and this is the only way they can get something working in a reasonable time. But then the student moves on and the result is something that may work, but nobody can build upon. Still, the situation you describe is unstable because as soon as the budget starts to look uncertain that engineer is likely the first to go. I really wish institutions would take this problem seriously and set up e.g. a department wide development group, which could help build and maintain stuff like that and ensure continuity. Unfortunately this seems to be against the current trend of cutting costs. For example, at the university I used to be, they cut the small common electronics shop, which was a great resource since the guys there knew all about who had what equipment, kept spare parts from broken stuff, etc. But in the budget it looked like pure cost I suppose. It's hard to measure the value of common resources and nobody wants to pay.
    • ThePhysicist 4y ago
• DeathArrow 4y ago
  You need to be very good, knowledgeable, always willing to learn and research, hard working and you make 50% of what a guy who did a 3 months bootcamp makes for working at a Rails crud app. Where do we sign up?
• smolder 4y ago
  Need a software engineer? Hire me, then. My last employer was charging $150 an hour for my time, and I'd say most clients felt it was a fair deal, or a good one. I quit to do other important things with my brief time on earth, and now that I'm looking for a software dev role again, it seems I'm hardly worth a third of that.
• gsliepen 4y ago
  I was a research engineer for ~10 years, and my experience was that my immediate colleagues were very appreciative of the work I was doing, but that the university bureaucracy system is not. In the beginning it is a very decent position (compared to a PhD); less load and better pay, and lots of autonomy. However, it is a dead end for your career, there is no promotion path. I liked working as a RSE a lot, but indeed moving to the industry will give you a huge salary rise and a career path.
• H8crilA 4y ago
  They will have more than enough once/if the tech valuations melt down and with them the software engineer salaries.
• TruthWillHurt 4y ago
  Well, it's a bit difficult when these positions require a degree. It's a problem with research institutes that they consider themselves academics and look down on people with extensive work experience.
• atemerev 4y ago
  I am a research software engineer, and I worked short-term contracts only, and my last contract was not prolonged due to budget cuts (with a laughable salary about four times less compared to industry — thankfully, I wasn’t there for money). The administrative personnel, on the other hand, were all working on permanent contracts. I guess science does not need more research software engineers after all!
• physicsguy 4y ago
  So, I was an RSE? Know why I left? Money. It’s all very well being an RSE but there are two ways of being funded (at least here in the UK). One is centrally funded by the University - universally this means being shoved into the IT hierarchy which is not a good fit. The other is on an ongoing basis by doing 1-2 year contracts. So no job security. The salaries for both of these are very poor. On leaving I took a 35% pay rise and do basically the same work but in industry. The only difference is that instead of working for engineers in academia, I do work for engineers in companies. Since I jumped, I negotiated another 20% pay rise. If I’d stayed, I’d have got a 3% pay rise. That’s not including the fact I get bonuses, private health insurance, don’t have to pay to park at my own office… The issue is that the skill set demanded to be an RSE is highly desirable in industry - able to write code well, often mathematical, familiarity with much of devops type work. On top of that, researchers don’t truly value your contributions. They don’t want to put you on papers that couldn’t have happened without your work. On top of that, they’re hugely demanding about timescales that are unrealistic. I got put onto one project which had contractual commitments to deliver to a major governmental agency to run a live service, and the guy running the project handed me a 5000 line Python 2.7 script that wouldn’t even run and basically said “get it working”. He couldn’t even supply the input files needed to run it for 2 months after that, just ignored emails. I quit before it got any further. Edit: forgot to add, my boss was a grade higher than me on the national academic salaries scale. He had technical responsibility and also had to line manage 10 people. Even if I could have gotten promoted (impossible - no budget), I didn’t want the line management responsibility. I earn more than the top of that salary band now, and line manage nobody.
• jacquesm 4y ago
  Then they'll have to pay market rates. But that in turn might cause the scientific staff to revolt.
• laurencerowe 4y ago
  > What do you envision for the profession over the next 10 years? > I want to see RSEs as equals in the academic environment. Software runs through the entire research process, but professors tend to get most of the recognition and prestige. Pieces of software can have just as much impact as certain research papers, some of them much more so. If RSEs can get the recognition and rewards that they deserve, then the career path will be that much more visible and attractive. As a former RSE I think this is unrealistic. Academia just isn't structured to value non-professors (it's far from just RSEs that are undervalued.) For me it wasn't so much the money but the refusal to sponsor permanent residence. When you see your amazing colleagues leave due to dumb departmental policies you realize you need to plan your own exit. I miss working in a team with a mix of scientists and engineers but I don't miss working for a university.
• tgbugs 4y ago
  For this to work academic science would have to start paying significantly more, which many labs simply cannot do, and other labs will refuse to do. My PI flat out said when we were trying to hire a python developer that they simply could not be paid 2x more than anyone else because it would cause major issues with team cohesion. 2x is the bare minimum to even get close to market rate for a full time developer who won't dance out the door a year later. By this logic either everyone in science needs to get a massive raise (stop to the PhD slave labor?), or rse's really have to justify their salary, and when everyone else works 60-80 hours a week for half the pay and is also defacto on call, that justification looks like zero work life balance. Talk to a funder near you, let them know that congress needs to provide more money for basic research. Be sure to sell it as training future employees for big tech too since they will all eventually leave after you finish teaching them.
  • cycomanic 4y ago
    Yes I don't understand the complaints about the salary a software engineer would be paid in academia. A software engineer straight out of undergraduate, typically earns as much (and often more) than a full professor at the university, who is >20 years after their undergraduate. Should they pay the software engineer more than everyone else, even though they don't even work on the core mission (i.e. the science)? What is even more ironic is that this comes from the same crowd who complains about high taxes, well research is primarily paid from tax money (private foundations/donations are a minuscule part of overall research funding), so the money to pay the software engineers would need to come from taxes.
    • joshuaissac 4y ago
      > Should they pay the software engineer more than everyone else, even though they don't even work on the core mission (i.e. the science)? They do not have to pay the software engineer anything if they can find one who is happy with getting nothing. Salary is always going to be a factor in who they can attract, especially when their competitors can offer much better compensation. If they are satisfied with the software engineers they are currently getting, then they can keep doing what they are doing now. What the professor gets paid matters very little compared to what the candidate can get at a competitor.
• deleted 4y ago
  [deleted]
• jacek 4y ago
  I used to work as a research software engineer in life sciences and I absolutely loved it. Jobs at universities allowed me to learn and experiment. However, universities cannot compete with industry when it comes to salaries. For example, in Germany where I now live, all public universities (and almost all prestigious schools that do high-profile research are public) have strict salary rules. You can find tables (like this one [1]), according to which your pay is determined. It depends on the job, land (state), degree, and years worked. While this kind of money allows comfortable life (or at least did before the inflation hit hard), industry offers at least 30% more (based on my perception). [1] https://www.jobs-beim-staat.de/tarif/tv-l_e13 https://www.jobs-beim-staat.de/tarif/tv-l_e13
• skohan 4y ago
  I would actually love to contribute to scientific research. I work in industry, which is lucrative and impactful but not that satisfying, and I work on pet projects which are satisfying but not that impactful. I would happily contribute some of my time to scientific projects which are legitimately impactful and also challenging.
  • haihaibye 4y ago
    What areas of science are you interested in, and what languages/skillsets do you have? Maybe I can steer you towards some Github repos
    • skohan 4y ago
      I am interested in biology and especially neuroscience, and I have a pretty broad skill set. I have a background in graphics/GPGPU programming and enjoy building simulations.
      • haihaibye 4y ago
        Maybe look into Molecular dynamics? https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6209097/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6209097/ Not an expert but this looks like the most popular repo: https://github.com/gromacs/gromacs https://github.com/gromacs/gromacs Good luck!
• somishere 4y ago
  Interesting topic. I recently finished a role that was essentially 50% RSE, but working at a university-adjacent non-profit. I found the academic work highly interesting+ enjoyable, but then most of the tools / applications I built were built based on personal experience / perceived gaps and then adopted, rather than at the request of a researcher. For the most part I found the tech resources available to researchers ad hoc at best, but generally non-existent or otherwise out of reach.
• signaru 4y ago
  Research software engineers need to be more recognized. In my workplace, I am the rare one who can code C++ to work with drivers for real time stuff and develop C# GUIs for convenience and automation. Everyone else is happy with Python or Matlab scripts that are only usable to one person and always need to be edited even for the simplest parameter changes. I would do analysis stuff in real time in a camera feed and adjust the experiment in real time, while others could only analyze an image after being saved. The difference in productivity is literally a hundred times. In most cases the job would have been impossible otherwise or at least painfully inefficient. Unfortunately, research leaders cannot tell the difference (or pretend to), and just label everyone the same. They are not software engineers themselves, and are probably even insecure. I feel my opportunities and potential are not being recognized either. I also feel that they don't respect that software development needs more time as they will just dump as much tasks to me as the other non-software engineers. I have to eventually leave such groups as I don't see their willingness to step up in their coding skills despite being heavily reliant on my output. Draconian intellectual ownership rules don't help either.
• milliams 4y ago
  I'm an RSE and the president of the Society of Research Software Engineering (https://society-rse.org https://society-rse.org), a small grass-roots registered charity in the UK that acts as a home for RSEs in the UK and internationally. We came about from the organisers of the RSE conference (https://rsecon2022.society-rse.org https://rsecon2022.society-rse.org) and from the early pioneers in the naming and recognition of RSEs. The other comments in this thread indeed point out the issue of poor salary compared with industry, this is a problem that I feel will persist. Historically, the benefit of working at a University over a company has been greater freedom, better benefits and (at least for me) the knowledge that you;re working for the public benefit. Unfortunately, the benefits are being reduced as time goes on as Universities are under greater financial stress. My role as an RSE at the University of Bristol has been in a primarily teaching position for the last few years, where I have been training PhD students, postdocs and research staff (all the way up to and including professors) in both the basics of programming but also testing, version control, profiling etc. My role is funded indirectly by the government as it is recognised by the funding agencies that RSE skills are essential for modern research and they are putting their money where their mouth is by funding more software projects, fellowships and training programmes in this area. If you think that you'd like to know more about getting involved as an as RSE, or if you work as one an would like to meet like-minded people then you can join as a member of the Society, join our Slack channel or our mailing list (https://society-rse.org/join-us/ https://society-rse.org/join-us/) or find out how you can get involved (https://society-rse.org/community/get-involved/ https://society-rse.org/community/get-involved/).
• iosystem 4y ago
  As someone that works as one. I would recommend NOT being a research software engineer if you value your health and haven't obtained financial freedom. Cost of living is always increasing and the lower pay isn't worth it. The work is more stressful as well because proper software development isn't respected. Previously worked at FANG.
• milliams 4y ago
  I think it's also worth clarifying that there are more than one style of RSE. The name came about to describe those researchers/postdocs working in research groups whose contribution was more to the software than to the writing of research papers. For them, the incentive structure wasn't there as they were being measured on papers written, not "research improved". By giving them the name RSE, it recognises that they are playing a different role in the group and perhaps need different metrics. At this time, they are still in research groups, experts in their field, having likely done a PhD in the particular domain. These are still RSEs and there are lots of them around (probably more than there are in central groups). The recent trend is for universities to provide central RSE groups (either by core-funding or being costed onto grants in parts) which operate more like consultants. This allows a route in for people who do not have a PhD or a background in a particular field, and can support many smaller projects across the university. For example at Bristol, we have a combination. We have a lose federation of "embedded" RSEs and some "RSE group" RSEs (funded through various grants) who work together to learn from each other, teach around the uni and provide a support network. This group was able to be formed due to a fellowship from the government back in 2015 (the same fellowship that allowed Paul to start his group at Sheffield). It's important to remember that anyone who is doing "research software engineering" is an RSE, even if it's not their job title.
  • physicsguy 4y ago
    Just wondering what you think about the progression routes for RSEs? A big problem as I see it is that someone not in the first wave of RSEs (who are now mostly in leadership positions across the various research intensive Universities groups) basically gets stuck with no appreciable career growth. You can probably get to a permanent job in the 42-50k salary band as a “Senior RSE”, which seems to be used for people even without that much experience to try and bump up the salary, but beyond that it would be very difficult to jump to the band above without staying in the same role for ~5+ years and hoping that someone left. At that point you’re obviously at top of automatic increment band and don’t get a meaningful pay rise for a potentially number of years. I thought it was very unlikely that there would more jobs at that level because when bidding for time on grants, academics don’t want to pay someone external to their team more than they pay their own post docs, and heard comments to that effect from academics themselves. For centrally funded positions it’s even harder, having seen my old group lead battling to try and get even 32-40k positions approved. So in my mind, there’s a big problem where someone ambitious could job hop into an easier and better paid job in industry. Why wait a really long time to get promoted when you could leave and get paid more than the top of the next salary band anyway?
• ajdegol 4y ago
  The issue with money is down to the university overheads. An RSE is paid like a standard researcher (with none of the benefits, recognition etc as detailed in other comments), which means that the university will take, minimum 50% of the grant money for that person. This is for building work, maintenance, general staff, etc etc. But I need a laptop, and I work from home, most of the time the IT staff just get in my way and slow my computer down with the crap they install. If the universities didn't take that cut, we could easily pay the RSEs more. There's no job security, very little recognition, but every single subject requires good code these days. The real issue here is that without decent coders our science will fall behind. Any idiot can write bad code, and smart people can write exceptionally bad code, we've all been there, not writing simple readable code because it made us feel clever, till we had to read it a year later. Academics are smart people who believe they can teach themselves to code, and they can, but it's generally not good software. There are also the ancillary benefits of just having someone around who can do things like rename your hundreds of files by writing a script in 5 minutes rather than you taking two days to do it (this actually happened). For the sake of western science this should be addressed.
• georgia_peach 4y ago
  Like looks after like. No matter how good you are, how much value you bring, if you don't fall under the employer's NAICS code, expect the waterboy treatment.
• dukoid 4y ago
  I wish there was a platform connecting research in need for some coding with volunteers. It probably wouldn't work for large scale projects, but for smaller problems it might be worth a shot, perhaps utilizing the stackoverflow platform?
• fancyfredbot 4y ago
  I think this doesn't frame the problem correctly. It promotes a separation between software engineers and real scientists which helps nobody. Anyone building software for scientific purposes really should understand the science behind it. Also, since better software allows better scientific outcomes, scientists need to understand software engineering. Both should be considered scientists and be on an equal footing.
• pdimitar 4y ago
  Responding to title only, and a little rant: No, scientists need to stop hand-waving away everything that's not their area as "trivial and not worth talking about" because yeah, I've heard that exact line from three separate individuals. When I shrugged it off and still produced tools for them that shaved off a literal 1-2h out of their day and giving them more time to do the actually creative things... they were so overjoyed they forgot to say thanks and immediately proceeded to act as if the tools were always there. They were extremely careful not to show any attribution or even basic human gratitude. Would it kill them? Yeah, frak that crap. A lot of scientists become severely tunnel-visioned, arrogant, conservative, downright disrespectful, and very hard to work with. Pile the extremely subpar payment and you really have to just be a bored college kid (or an adult with a huge financial safety net with a passion for the area) to engage with those people in a professional setting at all. I won't go into details about why I think many are like that because frankly, even if I was 100% correct (and my analysis is not at all flattering) it still doesn't matter one bit. Lasting change comes only from within. Scientists should start holding each other to a higher behavioral and collegiate standards and expel those who don't comply. Peer pressure works but many are so independent and without any oversight that they gradually fall under the illusion that their insufferable quirks are a personality or a personal touch that helps them excel in their area (and it's often the case that they don't excel in it). Looking down on people because they don't know your extremely niche physics research -- that has zero experiments attached to it, by the way -- is not helping your cause with people who usually get a lot of money AND respect AND a better working environment for actually improving your workday. In most cases these people will of course go where they are at least not looked down upon.
  • the_only_law 4y ago
    > A lot of scientists become severely tunnel-visioned, arrogant, conservative, downright disrespectful, and very hard to work with. Tbf I can replace scientist with developer and it’s hold just as true.
    • pdimitar 4y ago
      And you would be correct (and I am saying this as a programmer). Truth is, it's extremely easy for us the Homo Sapiens to become very set in our ways. :| I'm 42 and I've had colleagues at 28 year old that already act like they belong to a council of elders. Made me super sad... they are way too young for that!
• optimalsolver 4y ago
  Software engineer looks at some scientists' code and speeds it up 14000X: http://james.hiebert.name/blog/work/2015/09/14/CS-FTW.html http://james.hiebert.name/blog/work/2015/09/14/CS-FTW.html
• srg0 4y ago
  The principal incentive of science is publish or perish. Novelty trumps replication of the previous results (see replication crisis). Use of the citation metrics to evaluate performance in science is similar to relying on likes in the social media to measure importance. The number of citations matter more than the effort (cost) of producing the result. This environment favors those who can deliver something that's good enough, and then move on (and publish often). Software engineering is a continuous effort. Maintenance often requires more time and resources than the original development. It's more like curating a library or a database. Their product is perceived to stay the same. Incremental improvements do not advance the career of the maintainer at all. Researchers who sink their time into software development mostly hurt their academic careers. Even if they publish a paper about ResearchTool 1.0.0, they won't be able to publish another one about ResearchTool 1.0.1 and another one about ResearchTool 1.0.2. Meanwhile, their peers will probably publish "A", "B", and "C". Science needs a career path which does not depend on the number of publications.
• euroderf 4y ago
  Having a Ph.D sounds like an ANTI-qualification for this kind of position. A PhD would be a diversion from the kind of practical experience that would make a software developer with a sci/tech background & scientific interests useful in an environment full of smart people who possibly can't code worth a damn. More important than having one's own PhD laser-focus would be an ability to communicate with a variety of people doing their own research, and understand their domains just well enough to write/fix/improve their damn software and maybe even publish it (semi-productised) as open source too. A couple of application languages, some shell skills, maybe some data wrangling, and some software development common sense borne of experience.
  • epse 4y ago
    Possibly, but at least some experience in academia is terribly helpful, speaking from experience. There's a lot of particuliarities in funding, organisation, terminology, special stacks, ... You can certainly learn them "on the job", but if you already know them, even better
    • euroderf 4y ago
      Just wondering then, how much of that kind of knowledge is portable among academic research environments, and how much is peculiar to every individual environment ? Is there something that is dying to be written up for people entering the field ?
      • aragilar 4y ago
        Different aspects will be differently portable (e.g. funding will region specific, modulo field specific schemes). Having a PhD (in a relevant field) gives you domain knowledge as well as knowing where the pitfalls are, and a PhD in general will mean you're at least familiar with what questions to ask/who to ask those questions.
    • ehnto 4y ago
      I found this to be true. I was totally blow away by just how different of a world academia was, and how differently it operated. Software as a cost centre was a very unique experience on it's own, and software practices melded with institutional processes was genuinely shocking to me. Being able to take that stuff on the chin and still want to work there is probably not something you'll get from people who've been enjoying a cruisey experience in industry.
• deleted 4y ago
  [deleted]
• Tade0 4y ago
  I guess I fit this category, because I'm working on a project that's based off a scientific paper and is basically a Matlab script turned webapp so that all those calculations don't have to be done locally. The pay is indeed around 60-70% of what the industry offers, but I would have the same rate if was making a CRUD app[0] for the same large company, so it's more about the employer than the type of work. [0] which is something I did in the past - a webapp generating documents ensuring regulatory compliance. This organisation likes to make webapps.
• dwrodri 4y ago
  Computer Engineering PhD here. I’m seriously considering transitioning out of my program because I’ve reached a point where I’m just writing tons of code on a graduate student stipend and quite frankly, it seems doors are opening to me which I had previously thought would only be open with the PhD. I feel like my experience would be very different if I had one full-time research engineer who was paid a respectable sum to just... help me write code. I like working with undergraduate students, but I can’t do expect them to commit to the task like I would, or a full-time employee. They just don’t have the time. I’m doing my work extending / improving open-source CPU simulators and it’s just such a huge amount of code. I feel like my work would be so much more useful to others if I just had another pair of hands to work on docs, do ops work, go back and forth on decisions. My advisor doesn’t have the time for such things. - - - On the topic of academic pay: as a PhD student in Boston, I get paid $2000 a month pre-tax. Studios and 1BR apartments renting for under $1500/mo in a 30-minute commute radius of my campus are basically non-existent, so roommates and/or long commutes are a must. I’m realizing now why so few US citizens participate in graduate CS/CE programs: if you aren’t looking for a path to citizenship, it just doesn’t seem like a great way to spend your mid-to-late 20s.
  • hackernewds 4y ago
    this inadvertent details why the immigration path to citizenship is almost modern day labor exploitation. don't get me started on how the labor certification process for the green card furthers that in a timeline that is decades
  • iamcurious 4y ago
    Thank you, this is great information
• myst1 4y ago
  They should probably pay them the going wage for software engineers too. I used to work in science, the salary's are often half of what entry level swe positions are, and require an PhDs/post docs... People wonder why r&d in the us is in a slump... Well it's mostly because choosing a natural science for a profession is punishment compared to software.
• mr90210 4y ago
  Software Engineers need to be paid.
• qfwfq_ 4y ago
  I wonder whether this just creates another "underclass" of scientific laborers... I think we need tool-builders as academics. The only thing that academics recognize intellectually are peers. The rise of RSEs is a kind of intellectual outsourcing... implementation of your science should be part-in-parcel to its creation, not deferred to second-class non-academic "technicians." The friends and former students I've talked to who've entered RSE careers have largely been treated as second-class citizens in a research environment that they are integral to! I think, like many things, the buck stops with academia itself, its metrics, demands, and incentives. We need more research engineering/science about science academics within compute-intensive science departments themselves. Things like JoSS [1] or Scientific Data [2] are awesome first steps at addressing this. [1]: https://joss.theoj.org/ https://joss.theoj.org/ [2]: https://www.nature.com/sdata/ https://www.nature.com/sdata/
• caymanjim 4y ago
  I had the title "Research Software Engineer" well over 10 years ago, and worked as a research software engineer in another role prior to that with a random title. Contrary to many of the posts I see here, it was a great job, except for the below-market pay. I felt like I was contributing to something valuable. It beat working on yet another boring commerce website. I had interesting coworkers and worked with interesting technology. I didn't experience any of the politics around academia. I felt respected by everyone. I don't have a PhD. I don't even have an undergrad degree. I dropped out of highschool, got a GED, and did a couple years of undergrad, got hired by the university, and never finished. No one ever treated me like I was second-class. The only reason I'm not still working as a RSE is because I took a 50% pay cut when I took the job and was lured away by money later. I wonder how many of the negative commenters here have any actual experience as or with RSEs.
• msolo 4y ago
  I'm a Research Software Engineer with a PhD working in Harvard/MIT area for > 5 years. I skipped doing a postdoc and have been part several high-profile academic projects in genomics. Overall I've loved the experience and can be a great move for certain people. A major pain point in RSE is building and sustaining larger software teams around a a single, focused long-term project with ~6-8 RSEs working together. Even the most well-funded and successful academic labs have a limit to what how much RSE effort they can support. We need new funding models where engineering-focused leaders can muster sufficient resources (autonomously) to build great software teams that maintain high quality software in the long run. Focused Research Organizations (FROs) seem like an interesting new idea (https://www.dayoneproject.org/ideas/focused-research-organizations-to-accelerate-science-technology-and-medicine https://www.dayoneproject.org/ideas/focused-research-organiz...).
• dmead 4y ago
  Research software engineer here. This is probably not going to happen unless my employer decides to pay something competitive with industry. I'm here because I want to be. There is an opportunity cost here and not many people are willing to pay it.
• brnt 4y ago
  I work in an academic research software development team. We develop software that helps researchers execute neuroscientific models on HPC. Funding is our biggest headache. Currently we're funded through a (very) large EU grant, which gave our project a runway, but we find that nobody in general is funding such supportive (but essential) projects. Research funding is tied to scientific outputs (i.e. papers), and not anything else. We're trying to get some researchers interested in getting funding together, but it's not going very well, and we don't expect to be able to keep our current strength. If anyone has ideas, I'm all ears!
• micheles 4y ago
  I work as a research engineer in Italy. My path was academia (Physics)->industry (Finance)->mix. The "mix" where I am now is a strange case: it is located inside an university, nearly everybody has a PhD and we also have PhD students, but we are not academia, it is a private no profit company doing applied research. I am getting the same money I was getting in the industry but also 7+ weeks of vacation per year, more autonomy and less stress. Plus, I am not working to make rich people richer. Most of my job is about optimizing numeric code, lots of numpy, pandas, numba but also thinking of new algorithms. I get respect and they put my name in the research papers they publish, even if I did not write a word in the paper and have no knowledge at all of the underlying science ;-) Such companies are rare but they do exist. Much better than academia because there is no bureaucracy. I got in because of a friend of a friend...
  • iamcurious 4y ago
    Sounds heavenly, are they hiring?
• dontbenebby 4y ago
  No, they need to be willing to give a fair wage and benefits. I looked into scientific programming, and everyone I offered to work for if given an actual position, not an assigned seat and the perpetual threat of firing if a string of soft money grants ran out, and I got zero takers. The science field is full of performative narcissists who think if they're trying to "cure cancer" or whatever, the lessons from psychology, behavioral economics etc simply do not apply to them.
• asciimov 4y ago
  I spent a few years as a RSE in the mid-2010's. Supporting people doing actual science is a rewarding job. The worst parts of that job are the same for every body in research that isn't a PI pulling in lots of grant money: low pay, terrible or non-existent benefits, bureaucracy ran by non-science people, making do with older equipment. There is also a breaking in period where you have to prove you are not a moron. Since you come from a different educational area, and most likely aren't as educated, this can take some time and effort. The best part of the job: getting to work with highly educated people, getting to see science be discovered, being able to actually plan and execute projects because everything isn't on fire, regular business hours with no after hours support calls.
• kemiller 4y ago
  Most industries outside of the actual tech industry treat software engineers as low level technicians, pay and promote them accordingly, and then complain they can’t find anyone competent. Academia is that plus some of the worst internal culture you’ve ever seen. The work can be fascinating but unless you’ve already got fuck you money and just want something to do, stay away.
• mr_gibbins 4y ago
  The pay gap is real, and a huge reason why perfectly good software folk (and scientists, data specialists, etc.) move out of academia and into industry. I have an industry job with good pay with a side of academic work stemming from my PhD. My industry income is literally more than 2x the salary of my academic PI, who holds a senior academic post. My pay, which is probably around the median of most HN readers, is slightly more than the top peg of the academic pay scale in my area - i.e. University senior leadership team. So I do the academic work because I love doing the academic work. I cannot afford to take a full-time academic job, although it's been offered. I am baffled as to why academic institutions aren't failing to recruit. If they were businesses offering half the going rate, they'd get third-rate engineers, if they had any applications at all. Instead, there seems to be a long queue for academic tenure - and even for non-tenured positions. The only competitive advantage I can see is the sense of academic 'freedom' that these positions confer. But - it's not freedom. My academic colleagues spend hours each week chasing funding, filling in grant requests, attending meetings. Their teaching load is allocated to them without choice. They're in the office 9-5 same as everyone else. Academic freedom is anything but. I am lucky that for the research I do, mostly behind a laptop, my industry pay lets me afford the things I need to do it - cloud services, international travel, software licenses, article processing fees. I can 'do science' the way I want to without begging anyone for cash or going to grants committees. And I'm hoping this brand of 'citizen science' becomes the norm, because University-led research is going to get harder and harder the more the gap between industry and academic pay widens.
  • cozzyd 4y ago
    > I am baffled as to why academic institutions aren't failing to recruit Because academics (hi!) aren't in it for the money. If you value making lots of money, or having work-life balance, then academia isn't for you (nor is social work, teaching, nursing, fine art, etc.). But this probably makes it hard to hire RSEs who aren't academics masquerading as software engineers...
    • mr_gibbins 4y ago
      I agree, but there has to be a balance between doing it for love and getting a living wage. One of my colleagues, an Associate Professor, works in Halfords (shop/garage chain in the UK) on the weekends. That isn't right. And many of these folk are incredibly intelligent. They are Ricks amongst Mortys. They don't deserve to be paid not-quite-enough to afford a house and kids.
• dekhn 4y ago
  So much to say. I went through the whole academic cycle and moved to industry... about 15 years ago. I can't put it any other way than: the professors and PIs are abusing their employees. It has worked for some time because there were always enough people ultra-motivated to work in science for the prestige and other perks, but now, the economy is such that people can no longer do that (science job + family + mortgage = unhappy). The right way to think about this is in terms of long-term politics and the success of nations. If you disincentivize national research, your country will fare worse than its competitors. For me, that's the US- if the US doesn't turn around the incentive systems for doing high quality academic research in the national interest, we will eventually be at an economic disadvantage to China.
  • marysnovirgin 4y ago
    100% correct. The professors and PIs abuse their employees, and the institution completely supports it. It's the business plan.
• jleyank 4y ago
  (Very) few research groups can fund a commercial grade software engineer. Unless they’re intending to sell the software, it might not even be sensible to do so - like hiring another nmr specialist or whatever. These would be considered departmental support staff and groups could bid for their services. Chem departments used to have staff modellers for this, but the $$ in industry sucked them all away. Coding somebody else’s problem lacks status in academia, and the money is poor. Why would people do this as a career?
• waynecochran 4y ago
  I think it is funny that Perl saved the human genome project (https://bioperl.org/articles/How_Perl_saved_human_genome.html https://bioperl.org/articles/How_Perl_saved_human_genome.htm...). I wrote some Perl for a science grad student and it got me published in Genome Research: Matthew J. Lambert, Wayne O. Cochran. Kyle G. Olsen, Cynthia D. Cooper, Evidence for widespread subfunctionalization of splice forms in vertebrate genomes, {\em Genome Research.} 2015 May; 25(5): 624Ð632.
• harimau777 4y ago
  When the research wins a Nobel prize is the Software Engineer one of the winners or just the scientists in charge? My understanding is that it's the scientists. If Science wants more research software engineers then they should start treating them like peers rather than hired help.
• antipaul 4y ago
  “I want to see RSEs as equals in the academic environment“ Not just in academia, but also in biotech/pharma, where MDs or MBAs are too often the top dog. This largely explains why Pharma and healthcare is so behind when it comes to tech (Exceptions are the new, small biotechs. But the larger ones are more like dinosaurs)
• nspattak 4y ago
  Please do not hate me or hit me, I say these with an utter interest in the field. I worked as a research engineer for a few years. They are really needed but unfortunately this will be a waste of time, effort and knowledge. The real problem with software development in science is that it is driven by the scientists who are (mostly) short sighted in their sw dev planning/schedule, they do not appreciate software debt. And given their budgets and their focus in publishing, I understand their decisions.
• jnash 4y ago
  Sure pay me the same I am paid now and with the same benefits and I will be happy to do research. Your choice.
• rapjr9 4y ago
  I worked for 20 years at Dartmouth College as a "Senior Programmer" in the CS department, later reclassified as a "Senior Research Programmer". I came into the job with a BA in Physics and 15 years of industry experience (jamming systems for fighter aircraft EW systems, lead software QA programmer, consultant). While the pay was very low the work was interesting and towards the end I worked from home a lot. One difficulty was that my position was grant funded, so when a grant ran out so did my salary. The profs tried to keep me on several grants so I was always funded but at one point my primary boss went on sabbatical and I had to scramble to find someone to work for. This often meant I was working on several very different projects at the same time (computer security, privacy, robotics, wireless sensors, network monitoring, wearable health sensors, etc.), sometimes as many as 5 or 6 projects at the same time, usually 2 or 3. At one point I made an effort to collect information and build a proposal to create a pool of software+hardware engineers, similar to their already existing pool of programmers but more oriented to embedded devices, research code (the programmer pool mostly did statistics), wireless systems, drivers, etc. The College considered it, but decided not to go through with it because it would create a new cost center and they thought it would cost them money (benefits, what to do with the engineers between grants? put them on unemployment?). So unless a PI writes an engineer into a grant (difficult to do except for the larger $1,000,000 grants because you can't hire 1/3 of a person and get them to move to rural New Hampshire) there is basically no skilled engineering support for research projects at Dartmouth (except in the medical school). Most such work is assigned to students, who of course don't have a lot of experience and can't do some things (like write low level firmware for a Bluetooth MCU, CS students are not taught embedded systems coding, though they can take a course in the engineering school if they want to, but taking on a project that uses 500,000 lines of existing custom code and hacking it to do something no one has ever done before is not something a student can jump into for a semester.) So some types of research just don't get done. Also the work can be somewhat frustrating for an engineer because you never finish a project, you just get it working well enough to get a paper out of it and then it often ends. Even the longer term projects don't reach completion, at best there is a patent (I have my name on several as co-inventor, no one has ever licensed them.) There were very few other people like me on campus, I knew all of them and there were usually 3 to 5 at most. A year before I retired we hired a person to replace me, they stayed for a year then quit. It is very difficult to find people willing to move to a rural area, for lower pay, to do very complex unusual programming, that requires a wide range of expertise (from financial reports, to reviewing papers, to designing hardware, to all the kinds of software programming, collaborating with remote research groups, field experiments, travel to conferences, data analysis, cross disciplinary work in medicine, sociology, psychology, being comfortable in an international community, and more.) I think there are a lot of people who would enjoy this kind of work, but few willing to put up with the low pay and pressures (it's like being in a tiny startup that never grows and is always scrambling as they continually switch projects.) There was no path for advancement either (unless you call getting a PhD and leaving an advancement.) Pay raises were always fixed at about 2% per year (which actually meant losing pay since inflation incrementally reduced your salary until the next pay raise), or 3% at most in a good year. No bonuses. The benefits were good, the work was interesting, but from what I've seen it's difficult to find this kind of work at any research university, and it is extremely challenging in multiple dimensions. I think it likely this lack of engineers is /part/ of what has made much of US research short-term-focused on incremental changes and resulted in a lack of radical invention. Maybe it's different at some engineering schools?
• husamia 4y ago
  I started working at a top medical center 12 years ago. They didn't have a title for me. I picked one from the menu of current titles. Few years later, I asked to change it, two years later I got one. I feel like my title needs to be changed again.