3 ms·
For me, it depends obviously on why I’m reading it. If it’s a matter of background, I read the abstract and look at the figures to see if they’re reasonable.
by projektfu 6y ago
For me, it depends obviously on why I’m reading it. If it’s a matter of background, I read the abstract and look at the figures to see if they’re reasonable. Then I file it away.
If I need to design an experiment, I read the abstract and the methodology, then study the results to see if the methodology is any good. The references in the methodology often lead to other good papers. Here you find discrepancies in what they think they’re doing, what they tell you they’re doing, and what they’re actually doing. Reagents, for example, can be very specialized, and may not actually be available from the vendor they list. It’s a great way to “meet your neighbors”, asking them if they can help you find materials.
These days I mostly read critically, as I’m a practitioner and not a researcher. In this case I read the abstract and introduction. If the introduction isn’t 100% stuff I already know, I go out to the references. This makes critical reading time consuming and causes a fractal explosion of the number of papers in the collection. However, it is also how you find bullshit. Often someone in a review will assert that a reference supports a statement. Lazy or mendacious writers will copy that statement complete with the reference into their own paper. The actual reference may say nothing of the sort. Sometimes, there are multiple levels to this telephone game and each level loses nuance while adding error. When you see a lot of this, you know that there’s a bullshit problem in the domain.
If the author seems prone to BS or plagiarism, the results are somewhat questionable.
If the introduction is fine, I read the results and compare bits to the method. In my field, we look a lot at statistical power and effect sizes. Significance tests are often used incorrectly as a proxy and it pays to understand their limitations.
It’s also important to know how close the result is to the real world (for me). A paper showing reduced mortality, faster recovery, less pain, etc., is much better than one that shows a change in a biochemical marker or cellular protein and RNA content. A study in the species of interest is usually better than one in a model.
Ioannidis is right, most studies are flawed, so it’s best to avoid putting too much stock in any single one. A lot of good research has been done by questioning the accepted wisdom, which usually means one or two studies that may have flaws.