8 ms·

Oh look, another tiring craftsmanship debate that other disciplines long figured out! A, say, physicist writing bad code could equally well be building a pergo

• chongli 3y ago
  The real difference here is between hobbyists and mere workers. Programming happens to be one of the disciplines which has a lot of hobbyists. But that doesn’t mean you won’t find hobbyists in other disciplines. Look at machining for example. Long dominated by people working in machine shops making tools and parts on the clock. But in the background there’s a strong hobbyist contingent and there you’ll find endless debates over whether a beginner should buy a decades old Bridgeport milling machine or a brand new Chinese-made one. Similarly I find endless debates about what sort of frying pan to use (nonstick, stainless, carbon steel, or cast iron) among amateur chefs, but you’ll never see people working in a restaurant waste time on that. In other words, it’s hobbyists who really obsess over tools and craftsmanship. They do it because they love it. Programmers just happen to be among the odd sort who can get paid to practice their hobby.
  • esafak 3y ago
    I don't know about that. Pros obsess over it too, but they don't bother to endlessly talk about it; they just do it. But if a better tool comes along, they'll consider it.
• dash2 3y ago
  This might work for CERN. But a great deal of science is done by small teams who don't have a professional programmer available. Basically all of the social sciences, for a start; a lot of genetics too.
  • graemep 3y ago
    If you cannot model using decent code is it worth writing models at all? What if bugs mean the model is simply wrong? It has consequences too. There has been a lot of argument about how much impact the poor code quality of the Imperial college covid epidemiology model (which was the basis of British government policy during the pandemic) had on its accuracy. I do not know how bad it was, but it cannot be good the code was bad.
    • ben_w 3y ago
      One problem is that it's really hard to tell when you've just written bad code, which is also a problem for people whose job title is software developer, not just people who do it as a small part of their overall work. Some genes have been renamed because Excel interprets the old names as dates. The people who put all their genetic analysis into Excel had no reason to expect that, just as the people writing Excel itself weren't expecting the app to be used like this.
      • Cacti 3y ago
        Yeah but, in your example. this is a bookkeeping issue that, while frustrating and time consuming and costly, is just that. It’s not like a gas line in Manhattan blew up because someone in Toledo hit C-v in Excel. The scientists swapped around excel files and imported stuff without checking, which then was fed into other systems. A clusterfuck but one that is a daily occurrence at, minimally, every major non-tech company. It eventually gets unfucked with human labor or it simply wasn’t important in the first place. Exact same thing happens in research and academia. Point being, unknown unknowns are just that. But most unknowns are known and can be programmed defensively against for most serious use cases. All major fields are like this—like, you can hook up a car battery to light a menthol tank to boil two cups of water… or we can use a kettle. Perhaps for a brief point in time, due to our ignorance or just history, people lit containers of menthol on fire like it was sane, but that doesn’t mean it was, or is.
• ohlokkru 3y ago
  [flagged]
• tchalla 3y ago
  In computer science, for some reason - people struggle to distinguish between an engineer and a scientist.
  • hutzlibu 3y ago
    I think that is, because we are still at the frontier and the lines between research and developing something new are quite blurry. By now there are already lots of fields in IT that are quite standardized, but others not so much. For example, what is the fastest way to draw lots of shapes on a canvas on the web? There is no definite and fixed answer, as the field is still evolving and to find out the fastest way for your use case, you have to do research and experiments.
    • nottorp 3y ago
      > as the field is still evolving and to find out the fastest way for your use case No one in the world at large cares about the fastest way, they care about the lowest budget :)
      • hutzlibu 3y ago
        Depends. If gaming is what you do, the better the performance, the bigger the market. As then more people can play your game.
        • nottorp 3y ago
          HN is funny. I've been told on a previous discussion that AAA publishers don't optimize for potatos. Now you tell me it's a business requirement.
          • hutzlibu 3y ago
            I told you it depends. There are AAA games for console and gaming computers and there are casual mobile games for example. Very different markets.
• mihaaly 3y ago
  It is very true people should stick to the domain they know because otherwise they will have higher than average chance to f up. But that 'clear boundary' thing is a naive bollocks! No such thing! Both domain experts need to understand things beyond this imaginary and when precisely drawn then highly arbitrary boundary that is more like a gradient than a line normally (also not something relevant in a final good product). Teams consisting members unwilling to wander into foreign territory and expecting to be fed and deliver over a strict boundary will do horrible things! (we are not even trained this way btw., professions have quite a bit of overlap and we learn matters others are expected to take care of in practice, and in case of sofware engineering - apart from the most notorious ignorants no one wants to work with - it is pratically impossible avoiding to learn a big chunk of a foreign profession on the fly for delivering a good product, heavy science is no exception!)
  • 9dev 3y ago
    Perhaps I phrased this badly. My point wasn’t a clear boundary between professions, because you are right, that is difficult to impossible to draw. However, there’s a clear boundary between the goals of the code written by scientists vs. software engineers. Where a scientist aims to prove something, a software engineer builds code to produce business value. Both are trained very differently towards these goals.
    • LeonardoTolstoy 3y ago
      This probably highlights why so many software engineers in scientific R&D are "bad" in the strictest sense: they often don't understand that business value doesn't come from production-perfect solutions, but instead 100% of the business value comes from doing stuff fast, good enough, and understandable (so that researchers, not software engineers, can read it in a paper and iterate off of it).
    • mihaaly 3y ago
      I still disagree a bit. Their goals are the same: producing a product that fulfills the intended purpose and brings on value (financial or else or mixed). They may bring in their specialty learned beforehand (or even during) but this distribution of labour is not a goal but tool in reaching the common goal. Also no such as a software engineer is trained to seeks business value while a scientist seeks proof with clear cut separation, not at all.
• madsbuch 3y ago
  If the bad wood working would jepordize the results of his, professional salary earning, work then he should probably consider learning wood working, no?
  • 9dev 3y ago
    Don’t think so, no. A physicist has other stuff to learn and spend their time on. Instead, they should partner with a carpenter to do their woodworking from a rough sketch.
    • sidlls 3y ago
      If your point is that the physicist should partner with someone who is a "professional programmer" ("carpenter") to do the coding, I couldn't disagree more, speaking as a former research physicist who wrote many programs while I was in academia. A "rough sketch" is not enough for a "carpenter" to go off of for the programs a physicist using computational techniques is writing. They'd need to have a sophisticated understanding of the physics and the mathematical model involved: which almost no "carpenters" have.
      • 9dev 3y ago
        Now we’re lost in metaphors. A carpenter should definitely be able to build a pergola from a rough paper sketch, and a software engineer should be able to build a machine learning system from an algorithm paper by a data scientist.
        • sidlls 3y ago
          That really depends on what you mean by "machine learning system." For a business, implementing some researched algorithm and essentially using a template? Sure, it's possible. The research that goes into producing the algorithm in the first place? Probably not. There may be some classes of scientific programming which have simple enough models for a software engineer to implement. Scientific programming that relies on deep understanding of the domain and mathematical models employed to study it don't fit in that category.
• devjab 3y ago
  > It is a software engineer’s job to build quality software. It’s our job to deliver value to the business at a rapid and maintainable way. Rapid changes are often worth more to the business than maintainability, even over a period of many years. In some cases you could put a non software engineer down and let them build something with ChatGPT and it would work perfectly fine for the next 5-10 years because it’s focused on something particular, doesn’t change much and lives in isolation from the greater IT landscape. In other cases all your points are extremely valid. That being said we also work in an industry where a lot of “best practices” often turn out to be anti-patterns over a period of a decade. OOP is good in theory, and I know why we still teach it in academia, but in practice it very often lead to giant messes of complexity that nobody really knows how works. Not because the theory is wrong, but because people write code on Thursday afternoons after a week of no sleep and a day of terrible meetings. After a few decades in the industry, what I personally prefer isn’t any particular approach. No, what I prefer is that things are build in isolated services, so that they only handle their particular business related responsibilities. This way, you can always alter things in isolation and focus on improving code that needs it. It also means that some things can be build terrible, and be just fine, because the quality of the tiny service doesn’t really get to “matter” over its life time. I personally write code that follows most of our industry’s common best practices. Because it’s frankly faster once you get used to it, but I’ve seen really shitty spaghetti code perform its function perfectly and never need alteration in the 5-10 years it needed to live before being replaced.
• nohuck13 3y ago
  "It’s the same story, really: It is a software engineer’s job to build quality software. A scientists job is to solve problems." That's not the distinction. Good software engineers solve problems. That's what the paycheck is for. The distinction is whether code has to be maintained. It's the scientist's job to solve a specific problem at a specific time. Who cares if the metaphorical wood rots next winter, the paper's been published. It's _often_ the software engineer's job to build things that deliver business value over years, evolving and expanding requirements, in a development team, without grinding to a halt under the weight of accumulated complexity. "Quality" software engineering is just heuristics for keeping the pace of change high over time without breaking things.
  • zelphirkalt 3y ago
    > It's the scientist's job to solve a specific problem at a specific time. Who cares if the metaphorical wood rots next winter, the paper's been published. Sounds a little like cargo-culting than proper reproducible research. But this is a pest in academia definitely. Many papers do not provide all required data, all required model parameters etc. to get to the exact same result. Admittedly, they might nowadays need a software engineer to get that done.
    • jltsiren 3y ago
      Cargo-culting is about focusing on the process without fully understanding its purpose. Such as bureaucratic requirements for providing all data, software, parameters etc so that someone can reproduce exactly the same numbers with minimal effort. Proper reproducible research is not like that. It's about providing sufficient details that other people in the field can extrapolate the rest. That they can use similar methods with similar data to achieve similar results. Reproducing exactly the same result is not that valuable, as the "result" could be just an artifact of the specific data and specific methodology. Real validation depends on fully independent replications, with as little reuse of data and code as reasonably possible.
      • hytfyiv3j 3y ago
• kkoncevicius 3y ago
  The first distinction here should be what kind of "code" are we talking about. As far as I see there are two main possibilities: 1. code for performing a scientific simulation or analysis (a script). 2. code for solving a specific problem generally (a program). There are different "best practices" for the two situations above. And the article primarily talks about applying "best practices" from the 2nd scenario to the 1st. Of course they don't apply.
• jampekka 3y ago
  I'm an ex-software developer/engineer and current scientist. In my experience TFA makes a good point, even though it's quite strawmanish. Most scientific code is horrible from any sane software developer perspective. The quality is so bad that I think a huge proportion of published results are plain wrong due to bugs in the analysis. These apply to my much of my code as well. But a lot of "software engineering" code is horrible too. Mostly because most popular technologies and "best practices" are just plain bad. Overenginering is a pandemic and has been a long time. Much of the roots is from the gilded age of enterprise Java. Totally misunderstood OOP. Byzantine layers of pointless abstraction. Rampant premature "web scale". Counterproductive bondage and discipline (yes, including much of static typing). These have become a cargo cult in software development, and these cosmetic features are deemed as "quality code". And this leaks into scientific programming. Even Python, arguably the language of science nowadays, and a "quick and dirty one", forces some of this cargo cult. Modules are needlessly complicated (e.g. relative and absolute imports are quite a mess), let alone the horrible packaging system. And the current trend to push typing. That said, scientific code is getting slowly better, largely due to switch to public/open source code, and away from specialized hacks like MATLAB and R. Especially in more technical fields. In software engineering OTOH things are IMHO getting worse.
  • collyw 3y ago
    I'll add that "code reuse" was pushed as an idea too heavily (at my university at least). I think that leads to a fair bit of the over engineering and unnecessary complexity. I saw a comment later saying not to aim for "code reuse", but rather to "avoid code duplication" which made more sense. I also agree that a lot of best practices are great if you are a massive company with unlimited resources, but they are just overkill and add complexity for smaller projects and teams.
    • jampekka 3y ago
      Code reuse was indeed the hype, especially of OOP and inheritance. In practice it just made real code reuse a lot worse. The class hierarchies become so tightly coupled that any "reuse" requires a lot more boilerplate than the actual code to use. I know that the idea is not just not writing more code, but have a "single point of truth". But the boilerplate leads to a situation where it's very inpractical and leads to a tightly coupled mess that's really hard to change. What actually increased code reuse was duck typing (i.e. implicit interfaces), made popular by Python. But that's becoming verboten nowadays. And I don't think current structural typing systems are gonna reach the same level of reuse.
• flohofwoe 3y ago
  I guess the main problem is that many self-proclaimed "software engineers" are surprisingly bad "programmers" ;)
  • palata 3y ago
    Just like most "research" is absolutely useless, when not counter-productive ;)
    • collyw 3y ago
      It's often very useful for those funding it.
      • palata 3y ago
        Universities? Their main goal is to get better in the rankings, isn't it?
• barnabee 3y ago
  The problem is that most “professional” developers — i.e. people who write software as a career — are terrible at those things. Even (especially, sometimes) those that call themselves Software Engineers and talk endlessly about the right way to do things. That is not to say the kind of software engineer that does what you say and tends to build quality software (or at least move it in that direction) doesn’t exist, but the demand for people who can make computers do stuff (loosely, developers) so far outstrips the supply that outside of a few bubbles (HN being one of them), they appear to be so vanishingly rare they might as well not exist. I’m sure some scientists get to work with useful and highly valuable professional developers, but I’d be amazed if they were the majority.
• LeonardoTolstoy 3y ago
  >Neither does a scientist have to build proper software, nor does a developer have to do cutting-edge research. Sounds like the mission of Research Software Engineers (https://society-rse.org/ https://society-rse.org/). I work as a software engineer (with a PhD in a field that is not CS) in a research setting, and there is a give and take. 50% of my job is reading, understanding, and adapting very bad code from non-software engineers into a production system. But another 50% is binning the overwrought inflexible code written by my software engineer predecessors in order to do all that more quickly than refactoring would allow. In a research setting, in my opinion, MVP is king. Researchers seem to usually not produce viable long-term solutions. But software engineers do as well by virtue of not being domain experts (how could they be, they would need a PhD to understand the research domain!) and being unable to test 100% of the assumptions underlying the software themselves. Which is why it can help to have someone in between who is a domain expert, but knows just enough software engineering to produce production-good-enough code.
• dimask 3y ago
  Research software engineering is a hybrid field and fixed roles do not work well. It requires one having a grasp (at some level) of a lot of different stuff, eg software engineerin, the relevant scientific theories, statistics _and_, quite importantly, the culture of scientific practices in a field, in order to make something good. So it gathers a lot of people who, no matter where they started from, often have to sort of converge by learning stuff outside their own discipline. The problem is not that "physicists write code". Scientists end up learning a lot of stuff and getting good at it (software engineers the same). The problem is that writing software is often left as a job for the occasional phd, postdoc or research assistant, ie people with temporary positions, and in general to people who see building software as a side duty at best, annoyance at worst. This results in no generational knowledge building, being hard to find mentors, less learning and reflecting on practices on how to build software, rediscovering the wheel constantly, too much effort put into . It is not that one graduates as "software engineer" or as a "scientist" and then they know the best practices and everything of their respective fields. People get to learn stuff. Software engineering practices should be in the culture of scientific software building, not simply carrying them from the software engineering world to science, but adapting them and taking it each own idiosyncrasies as a field.
  • caddemon 3y ago
    I agree, a lot of unmaintained code is due to turnover of staff and few considering the software as an important output in its own right. But I don't think it would be that hard to find grad students/postdocs who would care about code, if that were something the field properly incentivized. Not only does software contribution not check the right boxes for career progression, it is also often looked down on. I find this especially laughable in biology... I've seen some (faculty) PhD committee members object to the student having a thesis chapter related to software contributions, because this is not "intellectual". But they are perfectly fine with one of the chapters being a wet lab paper where the student was a 3rd author who contributed purely through helping run experiments designed by the 1st author (e.g. handling mice, pipetting shit). There are PIs that simultaneously hold these two views, which to me signals a real misunderstanding of the challenges in and importance of writing decent software.
• rramadass 3y ago
  > It is a software engineer’s job to build quality software. A scientists job is to solve problems. There’s a clear boundary here, where the latter will deliver a concept to the former, who will eventually create a production-grade implementation off of that. Neither does a scientist have to build proper software, nor does a developer have to do cutting-edge research. Precisely! See my relevant comment from another thread here - https://news.ycombinator.com/item?id=38821679 https://news.ycombinator.com/item?id=38821679 References: 1) Why science needs more research software engineers - https://www.nature.com/articles/d41586-022-01516-2 https://www.nature.com/articles/d41586-022-01516-2 2) Research software engineering - https://en.wikipedia.org/wiki/Research_software_engineering https://en.wikipedia.org/wiki/Research_software_engineering
• deleted 3y ago
  [deleted]
• hax0ron3 3y ago
  >It is a software engineer’s job to build quality software. I think that most software engineers' jobs is not to build quality software, it is to make money. When you are trying to make money, the goal is not necessarily to make the best quality software that you can. Often, it is to make acceptably good software as soon as possible. A company that writes software that is half as good and ships it twice as fast might outcompete a company that writes software that is twice as good and ships it half as fast. I sometimes wish that my job was to build the best quality software that I can, but that is not the case. What I really get paid for is to make my employers' company successful, to make them money. And it's not that my employers don't care about making high quality software, it is just that if they cared about it too much they might get outcompeted by others who care less about it.