8 ms·
How to support open-source software and stay sane
- dddddaviddddd 7y agoMaintenance is a challenge anywhere where software is developed in-house without a dedicated development team. Development is often lead by one person and becomes very difficult when they depart. It seems like all the regular maintenance challenges are present in these situations, just exacerbated. Not sure what organizations that aren't software-focused can do to improve their situation in this regard.
- deleted 7y ago[deleted]
- PascLeRasc 7y agoOne thing we can do as users is champion the idea that open-source authors don't owe us anything. Having support or getting help with problems is great, but the author's already done us a huge favor by writing the software we needed in the first place, and they aren't required to go beyond that or do anything specifically for an individual.
- devxpy 7y agoExactly why the archive button exists :)
- crispyambulance 7y agoMostly agree, however, at some point the author's DO NEED to do something beyond creating the thing or else face the extinction of that piece of software. I think most people who have created something will generously bend over backwards to help individuals in the early stages of it's lifecycle. You can see that all the time on github. The problems come when the project takes off to the point where there isn't enough support for the number of people using it BUT the software isn't mature/popular/fit-enough to be "under the wing" of a larger organization who can afford to pay for it's maintenance and evolution. Is there a way to bridge the gap between author's-generosity-support and corporate/organizational stewardship? We do have the social networks in place to allow that, they're just focused on different objectives.
- SpaceManNabs 7y agoWasn't there a thing recently were an author just gave away one of his node.js libraries, and then it was used maliciously by the requester to attempt to hijack bitcoin wallets? Found it: https://arstechnica.com/information-technology/2018/11/hacker-backdoors-widely-used-open-source-software-to-steal-bitcoin/ https://arstechnica.com/information-technology/2018/11/hacke... I don't blame anyone in this scenario because the culture of open source projects and their interplay with enterprise encourages it
- TallGuyShort 7y agoYeah this is what I expected the article to be about: what drives me insane as an open-source developer is how a paying customer who has an outage at the worst possible time will be so much more polite and grateful for the help I'm contractually obligated to give them, than so many random people on forums are about the product not having a feature they want. It makes me want to give my customer's the source and tell them they can do whatever they want, and then ignore the rest of the community except for high-quality pull requests.
- ddavis 7y agoI think an answer to your final comment is quite simple: to invest in teaching proper software engineering practices, especially if it's not your focus. Get one or two people (potentially outside of the group/collaboration) that are experts and teach the group. I can say in high energy physics the environment is very much moving in the right direction. My collaboration has a dedicated tutorial three times a year which includes tutorials for things such as git and CMake (an overview of the concepts of version control and build systems are introduced as well, along with the definition of what a software release is). Just a few years ago this didn't exist; if you wanted to be proficient with these tools or understand the lingo you had to be self taught and a lot of people don't have the time to do that, so when they had to do it, it's like pulling teeth for a lot of people. Spending 2 to 3 days of a week 3 times a year is not a super serious commitment, and the material from the previous tutorial is always available for the next (always with some minor improvements/fixes). It gets people to a productive state a lot faster than just giving them a problem with our massive software stack and saying "oh and if you don't know git google it." I strongly believe all research groups that use software need well defined teaching material for how to be a productive user of and contributor to the local software stack. It's not a hard problem to solve and helps eliminate what are actually fake hard problems.
- 0xffff2 7y agoI'm not sure how well this approach works on average, but I haven't had a great experience. I'm a software engineer supporting a research group that's mostly CS PhD's. I take any chance I get to teach good software engineering practices, but they mostly just don't care.
- ddavis 7y agoYeah I hear you and empathize with this. That is especially difficult to deal with, but I think the model I describe with the three-times-a-year dedicated event helps, because people can really direct their focus for those few days and it's not random factoids as they come up.
- kkylin 7y agoAs an academic, I think that's a great first step. But there are many structural issues beyond this: advisors and students both under pressure to do whatever it takes to get the paper out and move on to the next project, students needing / wanting to just graduate and move on, usual yard sticks of academic achievement behind in recognizing software as legitimate product of research, etc (). On top of which, lots of different kinds of software is developed in research institutions, most of which are just rapid prototypes but some do get used by many people everyday, and it's not clear one process fits all. If anyone out there has suggestions of useful resources, I'm all ears! () Those of us who care about software do try to work on all these issues, but progress is slow.
- johannes1234321 7y agoIn Academia lots of this software is built by doctorate or postdoc students with limited contracts - they are gone after a short while. In many other places the people writing software outside of IT departments are typically staying longer with the company. Question now is: which is better? In one case there is a single "God" in the other it's passed on for generations, while everybody mostly cares about their research and not long term maintainability.
- wlesieutre 7y agoAgreed, I've done some automation work at my job for stuff that has exploded in volume over the last few years and wasn't feasible to keep doing by copy/pasting stuff through Excel anymore. It's in Python. I've avoided any external dependencies, kept inputs to CSV files that can be made from existing Excel sheets, and the code is fairly well commented. But there used to be two people here who've written at least a line of python in their lives. Now it's just me, and if I leave I have no illusions that it'll be maintained. Best thing to do is write instructions for whoever will need to run it, and they can hope that they never need it to do anything new.
- avgDev 7y agoI work at a place almost like that and I know the solution but nobody wants to pay for it. We develop software in house. I am the single dev on staff, we have a few contractors that we use for legacy ERP system programming/maint./modifications. I work with people who have CS degrees, however, it is still hard for them to understand why I'm spending time on layered architecture instead of using some basic OOP. Thankfully, they trust me and allow me to do what is required. The BEST solution I see is having more than one full stack dev but then you are paying 2x more. Also, have some kind of standard and review any outsourced work. I picked up a legacy app after an outside contractor and it has been a disaster to work with. The source code provided was out of date, he could not produce the source code that was running in production. No naming conventions were used, literally everything was just generic names like command1, textbox1, and so on. This could have been easily caught if any competent junior looked at the code. Some methods with tens of if statements, methods that are 1k+ lines long, almost zero OOP. If a company does have a single dev and cannot afford another they really need to stress maintenance and verify in some way that the dev is capable of producing a maintainable project. Therefore, they maybe should hire someone to help with the hiring process but hiring SEs is difficult even when experienced people are doing the hiring.
- zdw 7y agoDetermining worthiness of projects to fund is a really difficult battle. Do you go based on popularity? Importance? If something is worthy, how much is it funded? For how long? Who is paid to do the work? Even other projects that tried to address this like the Core Infrastructure Initiative seem to have unintended consequences. For example OpenSSL got CII funding, then used some portion of that to relicense as Apache 2 which breaks compat with the more free LibreSSL fork, weakening the overall community.
- dekhn 7y agoGreat article, thrilled to see PIs coming out and saying explicitly that the funding agencies are making a huge mistake funding discovery-driven science at the cost of long-term production work.
- xvilka 7y agoSpeaking about biomedical software - I suggested[1] to make a Julia flavor of Biostar Handbook[2], an amazing introduction into the field of bioinformatics and genomics from the Biostars[3] Q&A site authors. Porting algorithms to Julia will greatly improve the speed and maintainability of corresponding programs. [1] https://discourse.julialang.org/t/biostar-handbook-computational-genomics-and-julia-to-be-or-not-to-be/25732 https://discourse.julialang.org/t/biostar-handbook-computati... [2] https://www.biostarhandbook.com/ https://www.biostarhandbook.com/ [3] https://www.biostars.org/ https://www.biostars.org/
- deleted 7y ago[deleted]
- FpUser 7y ago"...Scientists writing open-source software often lack formal training in software engineering, which means that they might never have learnt best practices for code documentation and testing..." This is the most ridiculous statement. I had chance to work in either environment. Having that experience for the quality of the end result I will take scientist (preferably physicist or mathematician) with self taught software development skills over formally trained agile guru any time. Of course there are exceptions but ...
- oneepic 7y agoI have the exact opposite opinion, plus my cousin is a robotics/ML professor who has to deal with this same issue in his research group. Indeed, tons of scientists really have no idea about style, testing, etc. They're happy to just write an imperative C/C++/Python program with no docs whatsoever, run it, and be done with it.
- fghtr 7y agoWhy would scientists write good code with documentation if it is not recognized for their career?
- FpUser 7y agoI was talking about scientists who switched from science to writing software as a product for whatever reason. They surely produce good docs (properly documenting their work is the very basis of being scientist). As for writing imperative code in C/C++/Whatever other language they seem to choose: nothing is wrong with that.
- sbov 7y agoYou're basically saying scientists can become good programmers if they become professional programmers. Of course they can be, and I doubt anyone would argue that. Some of the best programmers I know have zero college education at all, so of course it's possible for mathematicians and scientists to do the same. But that isn't anywhere near the point of the article. It's talking about scientists that did NOT switch to writing software professionally, but continue to write it merely as a tool to accomplish their primary scientific goals.
- boron1006 7y agoGreat article, and fantastic to see a spotlight on an issue that I've thought a lot about. The sad part is that to a lot of scientists and researchers, software/software engineers isn't something worth paying for. It's not uncommon to see "programmer" jobs that are looking for 3+ years of experience that offer <$15 dollars an hour in the US. Sometimes they're "volunteer intern" positions. Of course the people who end up filling these positions aren't usually actual developers, so the software gets built poorly, eventually gets scrapped, and the cycle continues. Management also hasn't really evolved past the 90's. Non-technical scientists often want 100% of control and to make each decision, but don't want to spend any time on it. This means developers often have little to no specs to work with, but spend all of their time guessing about what the scientists want, and having to go back and fix everything after. >“That’s really the tragedy of the funding agencies in general,” says Carpenter. “They’ll fund 50 different groups to make 50 different algorithms, but they won’t pay for one software engineer.” This is the crux of my frustration. It's not even 50 different algorithms often. A lot of the time, 50 different research groups will be working on very similar programs, and none will be able to deliver a working version. Though the article mentions that research funding does exist, clicking on one of those funding pages and looking through their examples reveals that only ~1/10 of their websites are actually still active, and they aren't old sites. Again this goes back to the whole "scientists don't value software thing". I've seen scientists happily sign off on spending $20,000+ on hardware components that would usually cost <$100 to make, but balk at contributing $50 yearly to support open source. I got lucky that I managed to find a place where I get paid fairly, and my boss is actually technical and can manage tech projects well, but these places are few and far between.
- opportune 7y agoA lot of the points you bring up re related to cost. Here's the thing about cost, let's say it costs $100k/year to hire a good software engineer capable of writing scientific code (able to program and test complex algorithms, write HPC code, turn whitepapers into code), which might even be an underestimate depending on how benefits are paid out and the area. You can also fund 3 more grad students for that kind of money. The grad students will directly convert a PI's money into authorships while the software engineer's contribution will be only indirect, and likely take years to pay off. Plus, with only a single software engineer, there's a good chance you get unlucky and end up with someone clueless/lazy. You would probably need 3-4 software engineers to make a functioning team with best practices and hedge your bets against accidentally hiring someone who sucks. So now we're talking 10+ grad students. Open source software is a bit different because many labs can band together to fund things they find useful. But again there are still issues with cost-effectiveness. I'm guessing most lab contributors to OSS would want some sort of quid-pro-quo which may not be realistic for all OSS projects. And by funding OSS you are also funding competing labs' abilities to use the same features you use, which is good for science in general but not good for people's careers sometimes
- ylem 7y agoI think part of it is funding--but part of it is also recognition. In a number of fields, the development of scientific software receives little recognition compared to the science--even if the impact is large in terms of the discoveries it enables. So, for say tenure or career advancement, it might be "nice" for you to do it, but nowhere near as important as publishing high impact papers (at least in many fields). Especially if a graduate student or postdoc commits too much time to it instead of research results, they risk not being able to continue in their field (though they probably have more options if they decide to leave science to become software developers).
- jgamman 7y agoif people paid the price of a coffee for good software, post-docs could attach themselves to a research group and fund themselves by maintaining quality software products - freelance scientist FTW. the problem here is that lots of people want _other_ people to work for free. if you're not being paid, it's a hobby and you don't owe anyone anything. if the science system relies on free labour and refuses to support it, that's a very different conversation and the results are predictable.
- a3_nm 7y agoArgh, this article is going to give researchers the misleading impression that releasing code as open-source is complicated, that you need to maintain it, ask yourself difficult questions, be an expert in programming, find funding to keep it around, etc. But in my field, in most cases the source code is never released at all. That's a far bigger problem that not having support to use it. So fellow academics, please don't use this article as an excuse to not release your code. When in doubt, just push the thing to Gitlab as is, add a README that says "This is research code for paper X and it is unmaintained.", and disappear. It's not ideal, it's not the best way to do science -- but it's much better than not releasing your code. Related: the CRAPL http://matt.might.net/articles/crapl/ http://matt.might.net/articles/crapl/
- _dps 7y ago> But in my field, in most cases the source code is never released at all. That's a far bigger problem that not having support to use it. I agree but I think it's good for people to know that almost any moderately successful open project will attract whiners, complainers, entitled people, and in some cases outright abuse. From the perspective of society, having no open code is worse than having open code that's unmaintained. We're agreed here. From an individual contributor's perspective, opening yourself up to varying forms of whining and abuse "for the social good" (and not, say, for your tenure or publication count or whatever a researcher cares about in the moment) is a bigger problem than just sitting quietly on stuff you don't want to become a drain on your life.
- Bakary 7y agoWhat exactly are the whiners going to do if you just ignore them?
- _dps 7y agoLeaving aside the outright abusive, probably nothing. But how excited would you be to receive in your inbox "Your [beloved project] didn't solve my pet problem the way I wanted! This software sucks and the devs are lazy bastards who don't listen to the community". Then imagine getting one of these a month, a week, even a day if your project is very successful. It's of course nothing like actual abuse or threats of violence but it's an emotional toll that I certainly wouldn't want to pay. I have better things to do with my life.