4 ms·

As an engineer I don't really understand how the effect of these bad studies can be such a big deal. Doesn't science build off other science? When one scienti

As an engineer I don't really understand how the effect of these bad studies can be such a big deal. Doesn't science build off other science? When one scientist takes the conclusion of another study for granted (Example that X drug is effective), and then tries to build off of it with their own study (testing if A or B delivery method is more effective), doesn't it become obvious fairly quickly the second study is based on a bad conclusion of the first study? And if the author of the second study calls out the author of the first study, this type of Peer Review could be almost as effective as having more Peer Reviewed literature or study replication, which is what everyone is asking for.

Q6T46nT668w6i3m 9y ago
> Doesn't science build off other science? When one scientist takes the conclusion of another study for granted (Example that X drug is effective), and then tries to build off of it with their own study (testing if A or B delivery method is more effective), doesn't it become obvious fairly quickly the second study is based on a bad conclusion of the first study? Indeed. In fact, this is extremely common in biology. It’s possible that some fields need to confront the replication crisis (psychology, for example, is frequently cited and the subject of the OP), but I really don’t see this in my field.
- nonbel 9y ago
  What I've seen in bio research is that inconsistent results are always chalked up to some difference in the methods.
bzbarsky 9y ago
> And if the author of the second study calls out the author of the first study The "if" there is one big problem. What seems to have happened instead, in some fields, is that studies that would challenge the results of existing ones simply don't get published. This is partly a result of scientists self-censoring ("I must have done something wrong") and partly a result of journals not accepting papers that challenge "established science". But also, again for some fields, the assumption that other scientists build on the existing work may not be true. The worst-case scenario is when someone does a study, the results are widely trumpeted and applied to the real world, no one keeps track of how they actually perform in the real world (which may be difficult to do anyway, in fields like social psychology), and no one bothers doing any more studies in the area because it's "settled science"...

Q6T46nT668w6i3m 9y ago

> Doesn't science build off other science? When one scientist takes the conclusion of another study for granted (Example that X drug is effective), and then tries to build off of it with their own study (testing if A or B delivery method is more effective), doesn't it become obvious fairly quickly the second study is based on a bad conclusion of the first study? Indeed. In fact, this is extremely common in biology. It’s possible that some fields need to confront the replication crisis (psychology, for example, is frequently cited and the subject of the OP), but I really don’t see this in my field.

nonbel 9y ago
What I've seen in bio research is that inconsistent results are always chalked up to some difference in the methods.

bzbarsky 9y ago

> And if the author of the second study calls out the author of the first study The "if" there is one big problem. What seems to have happened instead, in some fields, is that studies that would challenge the results of existing ones simply don't get published. This is partly a result of scientists self-censoring ("I must have done something wrong") and partly a result of journals not accepting papers that challenge "established science". But also, again for some fields, the assumption that other scientists build on the existing work may not be true. The worst-case scenario is when someone does a study, the results are widely trumpeted and applied to the real world, no one keeps track of how they actually perform in the real world (which may be difficult to do anyway, in fields like social psychology), and no one bothers doing any more studies in the area because it's "settled science"...