4 ms·

Many of the "good practices" in software development are primarily meant to reduce the cognitive effort on the part of the person reading the code. I suspect i

Many of the "good practices" in software development are primarily meant to reduce the cognitive effort on the part of the person reading the code. I suspect it will be a downvotably unpopular opinion, but a typical researcher has a significantly larger cognitive capacity than a typical software developer. Consequently, a piece of code of certain complexity will look simpler, and be easier to manipulate for a researcher than for an average software developer.

patates 5y ago
It's always easy if you're the only one writing/maintaining the code. Also, if your job is writing code, I'd bet you'd have less difficulty manipulating spaghetti code.
codyb 5y ago
Huh? A researcher has greater cognitive capacity? That's a funny take. My impression was -> Software developers main focus is developing software so they spend a HUGE amount of time developing software, maintaining software, noticing patterns and bugs and pitfalls in software and thus they get pretty decent at writing software. Data scientists main focus is developing models, so they spend a HUGE amount of time developing models, tweaking models, finding data, and cleaning data, and write basic software to achieve some of those goals. I wouldn't expect Albert Einstein, Mozart, or Beyoncé to be some fantastic software developer just cause they're smart individuals. I'd expect people who spend a lot of time writing software to generally be the ones who write decent software.
naomisperfume 5y ago
This is a valid point, but I think in part the data science / machine learning cycle doesn't reward careful development like software engineering does. Most of the time you're just testing the viability of things and just want to fail fast if they don't work. Getting too attached to some pipeline or abstractions is usually a bad idea. I don't think this excuses bad code, but I kinda get it when it happens, since you can't just use best practices from the beginning like in software development
MengerSponge 5y ago
As a researcher, this is flattering, but probably wrong. Reading other people's code always sucks, but reading somebody's solo project hero code that doesn't follow any typical design patterns? That's nightmarish. I'd argue it's more about startup time, or whatever the technical term is for the time between when you first start reading code and when you've built enough of the model in your working memory to be able to touch it without breaking everything. That heroic developer doesn't need to concern themselves with other people's experiences, and they can just write whatever weird idioms suit their brain.
woeirua 5y ago
I think most research code bases are used by a significantly smaller number of people. Hence it's easier for the researcher to know what's going on in their code because they probably wrote a significant portion of the code they use and hence understand the assumptions, and limitations that are baked into the code. Whereas if you try to use a library normally you have to kind of figure those things out on your own. That said, having used a lot of research oriented software packages over the years, researchers typically do not produce high quality software. Their objective is to produce papers, not software that other people can use. If someone else can use it too, then great, but that's not the primary motivation.
deleted 5y ago
[deleted]

patates 5y ago

It's always easy if you're the only one writing/maintaining the code. Also, if your job is writing code, I'd bet you'd have less difficulty manipulating spaghetti code.

codyb 5y ago

Huh? A researcher has greater cognitive capacity? That's a funny take. My impression was -> Software developers main focus is developing software so they spend a HUGE amount of time developing software, maintaining software, noticing patterns and bugs and pitfalls in software and thus they get pretty decent at writing software. Data scientists main focus is developing models, so they spend a HUGE amount of time developing models, tweaking models, finding data, and cleaning data, and write basic software to achieve some of those goals. I wouldn't expect Albert Einstein, Mozart, or Beyoncé to be some fantastic software developer just cause they're smart individuals. I'd expect people who spend a lot of time writing software to generally be the ones who write decent software.

naomisperfume 5y ago

This is a valid point, but I think in part the data science / machine learning cycle doesn't reward careful development like software engineering does. Most of the time you're just testing the viability of things and just want to fail fast if they don't work. Getting too attached to some pipeline or abstractions is usually a bad idea. I don't think this excuses bad code, but I kinda get it when it happens, since you can't just use best practices from the beginning like in software development

MengerSponge 5y ago

As a researcher, this is flattering, but probably wrong. Reading other people's code always sucks, but reading somebody's solo project hero code that doesn't follow any typical design patterns? That's nightmarish. I'd argue it's more about startup time, or whatever the technical term is for the time between when you first start reading code and when you've built enough of the model in your working memory to be able to touch it without breaking everything. That heroic developer doesn't need to concern themselves with other people's experiences, and they can just write whatever weird idioms suit their brain.

woeirua 5y ago

I think most research code bases are used by a significantly smaller number of people. Hence it's easier for the researcher to know what's going on in their code because they probably wrote a significant portion of the code they use and hence understand the assumptions, and limitations that are baked into the code. Whereas if you try to use a library normally you have to kind of figure those things out on your own. That said, having used a lot of research oriented software packages over the years, researchers typically do not produce high quality software. Their objective is to produce papers, not software that other people can use. If someone else can use it too, then great, but that's not the primary motivation.

deleted 5y ago

[deleted]