4 ms·

I've yet to see an experiment software I've really liked. I've seen a few people use and struggle with OpenSesame recently, which seems like an incredibly featu

by Osmium 11y ago

I've yet to see an experiment software I've really liked. I've seen a few people use and struggle with OpenSesame recently, which seems like an incredibly featureful and useful project, but is also awkward because of it. Pyexperiment seems like a nice lightweight solution, but it has the same problem, which is that you need some scripting and computer skills. There's an argument which says that a modern researcher needs to be able to script/program, at least to a degree. But I don't like the idea of otherwise very able and very skilled scientists struggling to do good research because they're not great with computers. I'm not sure what can be done about this. On a large scale, I'd love to see proper investment into the UX and UI of existing scientific codes (and I have a long list of where to start with that). On a personal level, I'd really like to make an alternative to something like OpenSesame; not a replacement, but something more lightweight for smaller scale/student studies, with a nice GUI and easy to use. I wouldn't really know where to start though: what the essential features are and what you can leave out, because I don't do that sort of study myself.

semi-extrinsic 11y ago
> I don't like the idea of otherwise very able and very skilled scientists struggling to do good research because they're not great with computers. The solution already exists: skilled scientist hires grad student who's done some programming + some electronics classes. Happens everywhere already, works well. Talking about UI and UX as a problem with scientific codes makes me giggle though. We have much bigger issues with reproducibility, providence, mandating codes be open source, even getting people to use a version control system, that we need to fix first.
- Osmium 11y ago
  Interesting, would love to hear some of your experiences! It sounds like the issues we face are slightly different. From my field, version control is pretty much a non issue[1], most of the widespread codes use that (typically svn, but that's fine). Build systems are often an issue, as is incomplete (or outdated) documentation, and a lack of proper changelogs. A lack of proper tests too. Actually, maybe the issues we face aren't so different after all... > The solution already exists: skilled scientist hires grad student who's done some programming + some electronics classes. Happens everywhere already, works well. I've never actually seen this work well. Scripting an existing solution, sure. But writing a new one from scratch... Most scientists I have met personally who code, write awful awful code. Sometimes they just lack the time to do something better (I'm as guilty of this as anyone), other times they're just not very good at it. And it's almost never maintainable: when they leave the project, you might as well re-write from scratch (a more cynical person would say that's by design, but I actually don't believe that). There are many, many exceptions to this, obviously, but as a rule... I am optimistic though. It seems like things are changing for the better, slowly. Edit: [1] That said, I've been trying to persuade a colleague to use version control for ages. I'm at a loss. Live and let live, I guess.
  - semi-extrinsic 11y ago
    (What's you field? Mine's computational physics, but I collaborate a lot with experimentalists.) When I say version control, I meant for in-house projects, that tend to be just on someone's laptop and also a horrible mess, as you say. Most public codes are indeed on some VCS, git is very popular in my field. But reproducibility is a whole other matter. There's been some interesting work on providence systems that can attach all input needed to run a code for reproduction to e.g. figures in a paper, but until we've got a good universal one that journals start caring about, they're not going to see wide adoption. As for non-computer-savvy profs using grad students for programming: I see a lot of either LabView (for controlling experiments) or Python/Matlab in those cases, and it's often "write-only" code, but it mostly gets the job done. I'm also optimistic, but there are too few people working on and caring about the software carpentry we need to back modern svience in a good way.

by Osmium 11y ago

semi-extrinsic 11y ago

> I don't like the idea of otherwise very able and very skilled scientists struggling to do good research because they're not great with computers. The solution already exists: skilled scientist hires grad student who's done some programming + some electronics classes. Happens everywhere already, works well. Talking about UI and UX as a problem with scientific codes makes me giggle though. We have much bigger issues with reproducibility, providence, mandating codes be open source, even getting people to use a version control system, that we need to fix first.

Osmium 11y ago
Interesting, would love to hear some of your experiences! It sounds like the issues we face are slightly different. From my field, version control is pretty much a non issue[1], most of the widespread codes use that (typically svn, but that's fine). Build systems are often an issue, as is incomplete (or outdated) documentation, and a lack of proper changelogs. A lack of proper tests too. Actually, maybe the issues we face aren't so different after all... > The solution already exists: skilled scientist hires grad student who's done some programming + some electronics classes. Happens everywhere already, works well. I've never actually seen this work well. Scripting an existing solution, sure. But writing a new one from scratch... Most scientists I have met personally who code, write awful awful code. Sometimes they just lack the time to do something better (I'm as guilty of this as anyone), other times they're just not very good at it. And it's almost never maintainable: when they leave the project, you might as well re-write from scratch (a more cynical person would say that's by design, but I actually don't believe that). There are many, many exceptions to this, obviously, but as a rule... I am optimistic though. It seems like things are changing for the better, slowly. Edit: [1] That said, I've been trying to persuade a colleague to use version control for ages. I'm at a loss. Live and let live, I guess.
- semi-extrinsic 11y ago
  (What's you field? Mine's computational physics, but I collaborate a lot with experimentalists.) When I say version control, I meant for in-house projects, that tend to be just on someone's laptop and also a horrible mess, as you say. Most public codes are indeed on some VCS, git is very popular in my field. But reproducibility is a whole other matter. There's been some interesting work on providence systems that can attach all input needed to run a code for reproduction to e.g. figures in a paper, but until we've got a good universal one that journals start caring about, they're not going to see wide adoption. As for non-computer-savvy profs using grad students for programming: I see a lot of either LabView (for controlling experiments) or Python/Matlab in those cases, and it's often "write-only" code, but it mostly gets the job done. I'm also optimistic, but there are too few people working on and caring about the software carpentry we need to back modern svience in a good way.