4 ms·

Could we take it one step further? I'd love to see a publication format that bundles the code, the data, and maybe even the paper together into a single executa

Could we take it one step further? I'd love to see a publication format that bundles the code, the data, and maybe even the paper together into a single executable bundle for distribution. Imagine that the bundle starts with a manifest file that describes the data and the code, pointing to an authoritative, versioned copy. The user can "run" the manifest, which will download and install the data and any necessary dependencies. (If you're thinking about this from a containerized point of view, the code dependencies and data may be modeled as a container or virtual machine image.) The manifest is like a build file that executes the code passing the data as input, in a clear and reproducible way. Anyone running the manifest will get the same results as the researcher did. Because the entirety of the execution environment is versioned and captured by the manifest, including all of the system software, the results are reproducible from the code and data phase onward with zero effort. For bonus points, the manifest would also "build" the paper (e.g. if it's TeX), and would substitute the results of the code execution directly into the paper, i.e. graphs and numbers. You could conceptualize the paper as something like a notebook, where its values can change dynamically according with the data you provide it.

jackcosgrove 9y ago
Sounds like a Jupyter Notebook.
godelmachine 9y ago
Would all this be possible using git? In that case, even normal people (non-professors , non-R&D engineers) who read research papers to quench their curiosity, would have to know to use git.
- evincarofautumn 9y ago
  You could use an open protocol based on git, Docker, &c. internally, and build some interface friendly to non-programmers on top.
  - maxander 9y ago
    You would need more than a non-programmer friendly interface on top- biologists, astronomers, chemists, et al. usually aren't fully fledged programmers themselves, and aren't likely to understand what Docker is, let alone how to use it. A scientific programming project framework is a great idea (perhaps something built into Julia, or a specialized packaging of Python similar to Anaconda) but it would require a lot of the complex machinery to be concealed somehow. A challenging project, even before the "get editors to demand it" stage.
    - evincarofautumn 9y ago
      That’s exactly what I meant—conceal the details so anyone can use it, but base those details on common standards so it’s possible for a savvy individual to work with such packages manually without vendor lock-in. The true challenge definitely isn’t the technical side—it’s getting people to adopt it, and thereby make it standard.
cjbillington 9y ago
This unfortunately just isn't possible if you want your research to be able to be read in the future. "download?" from where? Links rot. "run?" which language, compiler, cpu architecture? Dependencies? Code bit rots. And data formats become unreadable - unless they're self-documenting like plain old csvs, they're fine. That's why algorithms are written in pseudocode, and mathematics is written in, well, mathematical notation. It's expected that if you want to use it you'll re-implement it yourself in a software setup convenient to you. So you can't publish runnable code unless it's severely limited - being in some standard language that is unchanging. I'd much prefer someone tell me in words what statistical method they used, so I can type it into Python myself than be forced to spin up some standardised but old and crufty fortran 77 or something that they didn't enjoy writing and I didn't enjoy running (I am aware that Python is often just calling the old fortran libs though!). Giving me the Python in the first place also isn't feasible - some scientific analysis code I wrote 6 months ago already doesn't run because of API changes in matplotlib. A few years ago I saw Python 1.5 code implementing an algorithm I wanted to use. I couldn't run it - instead I read it, and I read the Python 1.5 docs about the bits I was unfamiliar with. If it were in pseudocode it would have been more self-contained. Code and data formats, other than very crufty ones, are living. They're not suitable as the final form to publish in. If your project and field is ongoing, then by all means try to develop standards and use the same software and formats as each other to help sharing data. But the actual publications need to still be readable in a decade (or much more) time, so publishing runnable code seems like it conflicts too much with that.
- pacala 9y ago
  data => $0.1 / GB / Year in a public Cloud, or even $0.01 / GB / Year if using Glacier / Coldline. code => a Docker/CFI image. dependencies => captured in the image. format => the code understands the format.
  - pjmlp 9y ago
    That is short term thinking, who will guarantee any of that stuff will be around in the next 50 years?
cowpig 9y ago
http://www.gitxiv.com/ http://www.gitxiv.com/

jackcosgrove 9y ago

Sounds like a Jupyter Notebook.

godelmachine 9y ago

Would all this be possible using git? In that case, even normal people (non-professors , non-R&D engineers) who read research papers to quench their curiosity, would have to know to use git.

evincarofautumn 9y ago
You could use an open protocol based on git, Docker, &c. internally, and build some interface friendly to non-programmers on top.
- maxander 9y ago
  You would need more than a non-programmer friendly interface on top- biologists, astronomers, chemists, et al. usually aren't fully fledged programmers themselves, and aren't likely to understand what Docker is, let alone how to use it. A scientific programming project framework is a great idea (perhaps something built into Julia, or a specialized packaging of Python similar to Anaconda) but it would require a lot of the complex machinery to be concealed somehow. A challenging project, even before the "get editors to demand it" stage.
  - evincarofautumn 9y ago
    That’s exactly what I meant—conceal the details so anyone can use it, but base those details on common standards so it’s possible for a savvy individual to work with such packages manually without vendor lock-in. The true challenge definitely isn’t the technical side—it’s getting people to adopt it, and thereby make it standard.

cjbillington 9y ago

This unfortunately just isn't possible if you want your research to be able to be read in the future. "download?" from where? Links rot. "run?" which language, compiler, cpu architecture? Dependencies? Code bit rots. And data formats become unreadable - unless they're self-documenting like plain old csvs, they're fine. That's why algorithms are written in pseudocode, and mathematics is written in, well, mathematical notation. It's expected that if you want to use it you'll re-implement it yourself in a software setup convenient to you. So you can't publish runnable code unless it's severely limited - being in some standard language that is unchanging. I'd much prefer someone tell me in words what statistical method they used, so I can type it into Python myself than be forced to spin up some standardised but old and crufty fortran 77 or something that they didn't enjoy writing and I didn't enjoy running (I am aware that Python is often just calling the old fortran libs though!). Giving me the Python in the first place also isn't feasible - some scientific analysis code I wrote 6 months ago already doesn't run because of API changes in matplotlib. A few years ago I saw Python 1.5 code implementing an algorithm I wanted to use. I couldn't run it - instead I read it, and I read the Python 1.5 docs about the bits I was unfamiliar with. If it were in pseudocode it would have been more self-contained. Code and data formats, other than very crufty ones, are living. They're not suitable as the final form to publish in. If your project and field is ongoing, then by all means try to develop standards and use the same software and formats as each other to help sharing data. But the actual publications need to still be readable in a decade (or much more) time, so publishing runnable code seems like it conflicts too much with that.

pacala 9y ago
data => $0.1 / GB / Year in a public Cloud, or even $0.01 / GB / Year if using Glacier / Coldline. code => a Docker/CFI image. dependencies => captured in the image. format => the code understands the format.
- pjmlp 9y ago
  That is short term thinking, who will guarantee any of that stuff will be around in the next 50 years?

cowpig 9y ago

http://www.gitxiv.com/ http://www.gitxiv.com/