3 ms·

> How to integrate billions of base pairs of genomic data, and 10 times that amount of proteomic data, and historical patient data, and the results of pharmacol

> How to integrate billions of base pairs of genomic data, and 10 times that amount of proteomic data, and historical patient data, and the results of pharmacological screens into a coherent account of how somebody got sick and what to do to make them better? How to make actionable an endless stream of new temperature and precipitation data, and oceanographic and volcanic and seismic data? How to build, and make sense of, a neuron-by-neuron map of a thinking brain? Equipping scientists with computational notebooks, or some evolved form of them, might bring their minds to a level with problems now out of reach. The article seems to conflate the praxis of science with the archival of it. Scientists do all of the above on gigantic clusters, not on an IPython/Mathematica notebook. The purpose of publishing papers, on the other hand, is adding to the archival of knowledge, and they can be easily rendered in a laptop with LaTeX. And they are excellent at archival, by the way. You can see papers from the 19th century still being cited. On the other hand I have had issues running a Mathematica notebook from a few releases back -- and I seriously doubt one will be able to read any of my Mathematica notebooks 150 years from now. The same with the nifty web-based redesign of the Nature paper that is mentioned: I bet the original Nature article will be readable 150 years from now, whereas I doubt the web version will last 20.

tomkinstinch 9y ago
A group upstairs at the Broad Institute built out a system to use Jupyter notebooks for analysis of genomic data, with backend computation happening on a Spark cluster[1]. Science on large datasets can happen via interactive notebook. In a connection with a recent GWAS on a massive dataset from the UK Biobank, the researchers involved decided not to write a traditional scientific journal article (at least for now) since their analyses will continue to mature. Instead, they've been posting insights online in blog form, with associated code on GitHub[2]. It's a daring move toward publishing at the speed of research. Once their conclusions mature, traditional journal articles may follow to distill and preserve the key findings. In the mean time, those in the field can apply the same code to their data, replicate the analyses, and get an early look at the output of the research. This works partially because the methods (univariate GWAS) are understood in the field and the interpretation and rendering of a particular dataset is the science in this case, rather than a new method (which would still likely warrant a paper). 1. https://hail.is https://hail.is 2. http://www.nealelab.is/blog/2017/7/19/rapid-gwas-of-thousands-of-phenotypes-for-337000-samples-in-the-uk-biobank http://www.nealelab.is/blog/2017/7/19/rapid-gwas-of-thousand...
- JorgeGT 9y ago
  > Science on large datasets can happen via interactive notebook. I did not claim the opposite, just that it regularly happens without interactive notebooks. This seems like an interesting project though. Regarding the blog posts, it seems that there's a bug that makes all the entries appear as published on September 20, 2017?
akvadrako 9y ago
Of course people will be able to read your Mathematica notebook in 150 years. They will just load it into a 2020-era Mathematica-engine. The problem of software packaging will be solved by then, at least well enough to trivially emulate any of the popular environments we have today.

tomkinstinch 9y ago

A group upstairs at the Broad Institute built out a system to use Jupyter notebooks for analysis of genomic data, with backend computation happening on a Spark cluster[1]. Science on large datasets can happen via interactive notebook. In a connection with a recent GWAS on a massive dataset from the UK Biobank, the researchers involved decided not to write a traditional scientific journal article (at least for now) since their analyses will continue to mature. Instead, they've been posting insights online in blog form, with associated code on GitHub[2]. It's a daring move toward publishing at the speed of research. Once their conclusions mature, traditional journal articles may follow to distill and preserve the key findings. In the mean time, those in the field can apply the same code to their data, replicate the analyses, and get an early look at the output of the research. This works partially because the methods (univariate GWAS) are understood in the field and the interpretation and rendering of a particular dataset is the science in this case, rather than a new method (which would still likely warrant a paper). 1. https://hail.is https://hail.is 2. http://www.nealelab.is/blog/2017/7/19/rapid-gwas-of-thousands-of-phenotypes-for-337000-samples-in-the-uk-biobank http://www.nealelab.is/blog/2017/7/19/rapid-gwas-of-thousand...

JorgeGT 9y ago
> Science on large datasets can happen via interactive notebook. I did not claim the opposite, just that it regularly happens without interactive notebooks. This seems like an interesting project though. Regarding the blog posts, it seems that there's a bug that makes all the entries appear as published on September 20, 2017?

akvadrako 9y ago

Of course people will be able to read your Mathematica notebook in 150 years. They will just load it into a 2020-era Mathematica-engine. The problem of software packaging will be solved by then, at least well enough to trivially emulate any of the popular environments we have today.