6 ms·

I have a concrete counterexample. Let's say I write a paper presenting a model, plus some numerical results of large simulations. The code is based on gluing

by yakult 10y ago

I have a concrete counterexample. Let's say I write a paper presenting a model, plus some numerical results of large simulations. The code is based on gluing together various pieces of open source code. All these codes are typical scientist codes that are held together with duct tape. My paper is short, but I spent a lot of effort munging things together, and I'm fairly certain nobody can reproduce my results without my source code (preferably the whole environment) unless they spend a lot of time on trial and error like I did. The tweaks I did to glue things together has no theoretical value and don't belong in the paper. As a practical matter, I can't fit a lot of source code into short paper format. What do?

erichocean 10y ago
Open source the code and supporting data.
- yakult 10y ago
  It's not that simple. What if some of it is proprietary? What if I'm not allowed to submit code because I need to be anonymous so reviewers can maintain impartiality? What happens when one of the upstreams update and breaks my code? Do I need to keep it updated? Forever?
  - adrianN 10y ago
    At my institute at least, scientists are required to maintain everything that is necessary to reproduce a result for at least ten years. That includes all the data and the software used to produce the results. It's not an easy job, but it's important.
    - yakult 10y ago
      If your institute also mandates they make the data/software publicly available, then that's definitely the exception rather than rule. Also must be hideously expensive. It almost never happens that a paper I read actually comes with usable source code.
  - pixl97 10y ago
    Then your results are not reproducible and your conclusion is suspect.
deleted 10y ago
[deleted]
Klockan 10y ago
> Let's say I write a paper presenting a model, plus some numerical results of large simulations. The code is based on gluing together various pieces of open source code. All these codes are typical scientist codes that are held together with duct tape. My paper is short, but I spent a lot of effort munging things together, and I'm fairly certain nobody can reproduce my results without my source code (preferably the whole environment) unless they spend a lot of time on trial and error like I did. Then leave out the results since they are just an anecdote. If you want to include experimental results then it has to be done in a scientific fashion.
chriswarbo 10y ago
A lot of thought has gone into such questions. For example, see the guidelines at https://www.epsrc.ac.uk/about/standards/researchdata/ https://www.epsrc.ac.uk/about/standards/researchdata/

by yakult 10y ago

erichocean 10y ago

Open source the code and supporting data.

yakult 10y ago
It's not that simple. What if some of it is proprietary? What if I'm not allowed to submit code because I need to be anonymous so reviewers can maintain impartiality? What happens when one of the upstreams update and breaks my code? Do I need to keep it updated? Forever?
- adrianN 10y ago
  At my institute at least, scientists are required to maintain everything that is necessary to reproduce a result for at least ten years. That includes all the data and the software used to produce the results. It's not an easy job, but it's important.
  - yakult 10y ago
    If your institute also mandates they make the data/software publicly available, then that's definitely the exception rather than rule. Also must be hideously expensive. It almost never happens that a paper I read actually comes with usable source code.
- pixl97 10y ago
  Then your results are not reproducible and your conclusion is suspect.

deleted 10y ago

[deleted]

Klockan 10y ago

> Let's say I write a paper presenting a model, plus some numerical results of large simulations. The code is based on gluing together various pieces of open source code. All these codes are typical scientist codes that are held together with duct tape. My paper is short, but I spent a lot of effort munging things together, and I'm fairly certain nobody can reproduce my results without my source code (preferably the whole environment) unless they spend a lot of time on trial and error like I did. Then leave out the results since they are just an anecdote. If you want to include experimental results then it has to be done in a scientific fashion.

chriswarbo 10y ago

A lot of thought has gone into such questions. For example, see the guidelines at https://www.epsrc.ac.uk/about/standards/researchdata/ https://www.epsrc.ac.uk/about/standards/researchdata/