5 ms·

Yes, if anything we need better documentation of methods. This is something often overlooked with the open-scientific-data movement also: data on its own is not

Yes, if anything we need better documentation of methods. This is something often overlooked with the open-scientific-data movement also: data on its own is not very useful (in the sense of being insufficient to draw valid conclusions) if you don't have good documentation of how precisely it was collected, which is critical to interpreting it correctly... otherwise you just have some numbers in a CSV file. Traditionally PhD theses are one of the few places that really exhaustively documents what exactly they did and how, while papers tend to gloss over a lot of things. There's been a little noise about that [1], but not a lot of movement so far. I do think a movement of theses away from monograph-style to collection-of-articles style is okay, though. [1] http://scholarlykitchen.sspnet.org/2014/11/18/nevermind-the-data-where-are-the-protocols/ http://scholarlykitchen.sspnet.org/2014/11/18/nevermind-the-...

danieltillett 11y ago
There is no need these days for brevity with methods since everything is online. Put it all into the SI and be done with it. One problem with many methods is the lack of description of the pitfalls and critical steps. When you go to repeat an experiment you often don't have exactly the same equipment or reagents as the original paper. Most of the time this does not matter, but in an annoying number of cases some particular step needs to be performed using a precise piece of equipment or brand of reagent. It is only when you can't get the experiment to work and you contact the person on the bench who did the research that you learn "yeh we tried all the other reagents and only brand x works". If there is something critical put it in the method that it is critical!
- Thriptic 11y ago
  In my experience a lot of very critical steps in experiments are frequently never formally defined in literature or even within lab notebooks, existing only as institutional knowledge. What may seem like an obvious step not worth docimenting to the original investigators is non-obvious to people looking to repeat the work, leading to different results.
analog31 11y ago
My thesis was very much a recipe for how to make my experiment work. We frequently used the dissertations from former students as reference manuals for their methods. I cranked mine out in 3 months. Amusing anecdote: This was 20+ years ago, when technical word processing was still a work in progress. Many of my peers struggled endlessly with formatting. My plan was to create a mechanically rough manuscript, get through the defense, then worry about formatting afterwards. It worked, with a hitch: I never got page numbering to work, and ended up pasting the page numbers in by hand. Also, many of the figures are obviously pencil drawings that were photocopied.

danieltillett 11y ago

There is no need these days for brevity with methods since everything is online. Put it all into the SI and be done with it. One problem with many methods is the lack of description of the pitfalls and critical steps. When you go to repeat an experiment you often don't have exactly the same equipment or reagents as the original paper. Most of the time this does not matter, but in an annoying number of cases some particular step needs to be performed using a precise piece of equipment or brand of reagent. It is only when you can't get the experiment to work and you contact the person on the bench who did the research that you learn "yeh we tried all the other reagents and only brand x works". If there is something critical put it in the method that it is critical!

Thriptic 11y ago
In my experience a lot of very critical steps in experiments are frequently never formally defined in literature or even within lab notebooks, existing only as institutional knowledge. What may seem like an obvious step not worth docimenting to the original investigators is non-obvious to people looking to repeat the work, leading to different results.

analog31 11y ago

My thesis was very much a recipe for how to make my experiment work. We frequently used the dissertations from former students as reference manuals for their methods. I cranked mine out in 3 months. Amusing anecdote: This was 20+ years ago, when technical word processing was still a work in progress. Many of my peers struggled endlessly with formatting. My plan was to create a mechanically rough manuscript, get through the defense, then worry about formatting afterwards. It worked, with a hitch: I never got page numbering to work, and ended up pasting the page numbers in by hand. Also, many of the figures are obviously pencil drawings that were photocopied.