3 ms·

As discussed in the linked article, it's mostly because of code quality but with a healthy dose of percieved threat to job security. Unfortunately the academic

As discussed in the linked article, it's mostly because of code quality but with a healthy dose of percieved threat to job security. Unfortunately the academic citation economy isn't very nurturing of community codes, so a lot of work gets repeated when secretive research groups compete. That's not the whole story and the fact that this article is getting published speaks to that, however I still don't think it's likely that things will change in the short term. My friend Matt Turk wrote about a similar topic recently: http://arxiv.org/abs/1301.7064 http://arxiv.org/abs/1301.7064

zbyszek 13y ago
Repetition isn't necessarily a bad thing. Whether compatible results can be obtained with independent analyses of the same data is surely an important indicator of success. Of course that should not preclude the possibility of replication as described here. I find the idea of threats to job security misguided. In the particle physics world some significant code (not everything, granted) has been made available for some time; e.g. Monte-Carlo simulators like Geant or lattice gauge theory codes like those from MILC, SciDAC or FermiQCD. Even highly optimised code has been available. Users are requested to cite the authors. No careers have been harmed, as far as I can tell. Also, the code itself is not much use for replicating results without making the data available too. This can be more problematic politically and technically, but even here there are good precedents in the particle physics world.
raverbashing 13y ago
Unfortunately the world of scientific collaboration follows a strict pattern: The Article Which is good as a way of 'settling information' into a medium but an awful way for today. It's the equivalent of stone tablets compared to what's available today It would be much better if the product of research could be shared in a more flexible format. For example, don't get me started on reading two columns PDFs in a computer screen. Flexible formatting, attachable metadata, reusable "source code" (that could be an excel spreadsheet, a txt list of data, or source code per se), redrawable graphs, etc We need a faster, more flexible, albeit official way of sharing data. Yes, there are ownership issues, IP issues, etc. But maybe having a more easily searchable/traceable system may help with this.
- nilx 13y ago
  > We need a faster, more flexible, albeit official way of sharing data. Metajournals are trying to do it, with an "Article" overlay to data/software publication : http://metajnl.com/ http://metajnl.com/. IPOL also tries to publish image processing software: http://www.ipol.im/ http://www.ipol.im/. And there is also some work to interlink science assets (more than articles) on http://linkedscience.org/ http://linkedscience.org/. \ I welcome publication models not based on "PDF standard" (ie an electronic clone of the centuries-old ink&paper text article), but simply sharing data/software is not sufficient. I think it is important to have an editorial line and a peer review, otherwise anything online would qualify as scientific communication. Moreover, software and data is hardly sufficient by itself; it needs to be explained, documented, illustrated and discussed to be more than a "harddisk dump".
zer0gravity 13y ago
I think this is a deeper problem that affects the whole scientific academic community and maybe the solution to this, is the growth of a new parallel movement of amateur scientists that are pursuing research from passion and the desire to advance human knowledge, not because their job/position depends on it. In such an environment sharing your tools would probably be the norm.

zbyszek 13y ago

Repetition isn't necessarily a bad thing. Whether compatible results can be obtained with independent analyses of the same data is surely an important indicator of success. Of course that should not preclude the possibility of replication as described here. I find the idea of threats to job security misguided. In the particle physics world some significant code (not everything, granted) has been made available for some time; e.g. Monte-Carlo simulators like Geant or lattice gauge theory codes like those from MILC, SciDAC or FermiQCD. Even highly optimised code has been available. Users are requested to cite the authors. No careers have been harmed, as far as I can tell. Also, the code itself is not much use for replicating results without making the data available too. This can be more problematic politically and technically, but even here there are good precedents in the particle physics world.

raverbashing 13y ago

Unfortunately the world of scientific collaboration follows a strict pattern: The Article Which is good as a way of 'settling information' into a medium but an awful way for today. It's the equivalent of stone tablets compared to what's available today It would be much better if the product of research could be shared in a more flexible format. For example, don't get me started on reading two columns PDFs in a computer screen. Flexible formatting, attachable metadata, reusable "source code" (that could be an excel spreadsheet, a txt list of data, or source code per se), redrawable graphs, etc We need a faster, more flexible, albeit official way of sharing data. Yes, there are ownership issues, IP issues, etc. But maybe having a more easily searchable/traceable system may help with this.

nilx 13y ago
> We need a faster, more flexible, albeit official way of sharing data. Metajournals are trying to do it, with an "Article" overlay to data/software publication : http://metajnl.com/ http://metajnl.com/. IPOL also tries to publish image processing software: http://www.ipol.im/ http://www.ipol.im/. And there is also some work to interlink science assets (more than articles) on http://linkedscience.org/ http://linkedscience.org/. \ I welcome publication models not based on "PDF standard" (ie an electronic clone of the centuries-old ink&paper text article), but simply sharing data/software is not sufficient. I think it is important to have an editorial line and a peer review, otherwise anything online would qualify as scientific communication. Moreover, software and data is hardly sufficient by itself; it needs to be explained, documented, illustrated and discussed to be more than a "harddisk dump".

zer0gravity 13y ago

I think this is a deeper problem that affects the whole scientific academic community and maybe the solution to this, is the growth of a new parallel movement of amateur scientists that are pursuing research from passion and the desire to advance human knowledge, not because their job/position depends on it. In such an environment sharing your tools would probably be the norm.