2 ms·
There is certainly no problem doing a study and having a hypothesis that X is bad. There is a huge problem with going through the literature and selecting stud
by mikekchar 9y ago
There is certainly no problem doing a study and having a hypothesis that X is bad. There is a huge problem with going through the literature and selecting studies that agree with your point. Part of the problem lies in the fact that X is usually not completely good nor completely bad.
For example, let's say that 10% of all studies are poorly executed. If there are 100 studies that conclude that X is good, I can find 10 of them that are poorly executed. Let's say there are 20 studies that conclude that X is bad. If 10% of them are poorly executed, then I can find 18 which are well executed. In reality there are 90 good studies concluding that X is good and 18 good studies concluding that X is bad. However, I can show the 10 poor studies showing something is good and the 18 well done studies showing it is bad, leading you to conclude that there is some kind of conspiracy, even though most of the evidence concludes that X is good.
Meta studies are particularly prone to selection bias in this way.