3 ms·

Thanks for the links! Reading from a source and weeding out "low-quality stuff" is a valuable skill to have. Not so long ago a study on telomere length was lamb

by jmstfv 9y ago

Thanks for the links! Reading from a source and weeding out "low-quality stuff" is a valuable skill to have. Not so long ago a study on telomere length was lambasted by HN community for having low sample size [0] How do you choose which papers to follow? Any rules of thumb/red flags for assessing a paper? [0] https://news.ycombinator.com/item?id=14575361 https://news.ycombinator.com/item?id=14575361

goncalogordo 9y ago
Yes, definitely! First, I start with a clear goal/question in mind: e.g. which lifestyle changes could help me live longer? Then there are several things I look for when assessing a specific paper: a) Sample size is key. Small samples are not very reliable as you can reach almost any conclusion you want. I also prefer looking at meta-analysis (this is when scientists look at all the studies that analysed a given topic). This means more data (larger samples) and less risk of biases. b) Low P-value (below 5% is what you're looking for). This is the probability that the effect found was due to randomness. If this is too high, then you cannot rely on your conclusions. c) Adjusting for other variables that could explain the effect that was found in the study. E.g. if you want to find if eating more vegetables is healthy, you should adjust for exercise, so you can compare people that do the same amount of exercise but eat different amounts of vegetables. Otherwise the effect found could be explained by other variables not adjusted for. d) No funding bias. Is this a study stating that eating X is good but funded by the Association of X Producers? Avoid those :) e) No publication bias (this only applies to meta-analysis). This is when scientists don't publish their findings (e.g. because they did a small study, or found an insignificant effect, or found an effect they or their funders "didn't like"). Most good meta-analysis will comment on this (it's also called Funnel test/analysis). f) Human studies. I tend to ignore studies that aren't done with humans as the probability the conclusions will hold in humans is actually very low (probably less than 10%). It's also preferable to have intervention studies (when you get two groups of people that receive two different interventions - e.g. eating beets juice and placebo - and compare outcomes; they generally don't know which intervention they're getting) vs. observational studies (when you study a group of people and try to assess if differences in what they say they do lead to different health outcomes). But in this field, most studies are actually observational, hence the importance of all the other things I've mentioned. Hope this helps!

by jmstfv 9y ago

goncalogordo 9y ago

Yes, definitely! First, I start with a clear goal/question in mind: e.g. which lifestyle changes could help me live longer? Then there are several things I look for when assessing a specific paper: a) Sample size is key. Small samples are not very reliable as you can reach almost any conclusion you want. I also prefer looking at meta-analysis (this is when scientists look at all the studies that analysed a given topic). This means more data (larger samples) and less risk of biases. b) Low P-value (below 5% is what you're looking for). This is the probability that the effect found was due to randomness. If this is too high, then you cannot rely on your conclusions. c) Adjusting for other variables that could explain the effect that was found in the study. E.g. if you want to find if eating more vegetables is healthy, you should adjust for exercise, so you can compare people that do the same amount of exercise but eat different amounts of vegetables. Otherwise the effect found could be explained by other variables not adjusted for. d) No funding bias. Is this a study stating that eating X is good but funded by the Association of X Producers? Avoid those :) e) No publication bias (this only applies to meta-analysis). This is when scientists don't publish their findings (e.g. because they did a small study, or found an insignificant effect, or found an effect they or their funders "didn't like"). Most good meta-analysis will comment on this (it's also called Funnel test/analysis). f) Human studies. I tend to ignore studies that aren't done with humans as the probability the conclusions will hold in humans is actually very low (probably less than 10%). It's also preferable to have intervention studies (when you get two groups of people that receive two different interventions - e.g. eating beets juice and placebo - and compare outcomes; they generally don't know which intervention they're getting) vs. observational studies (when you study a group of people and try to assess if differences in what they say they do lead to different health outcomes). But in this field, most studies are actually observational, hence the importance of all the other things I've mentioned. Hope this helps!