3 ms·

This issue is true with much of social science. It would be great to see more replication (or non-replication) experiments to confirm what we think we do or do

This issue is true with much of social science. It would be great to see more replication (or non-replication) experiments to confirm what we think we do or don't know.

DanBC 12y ago
The linked comic is true - there are many problems with social science studies. But, since you specifically mention replication and non-replication -- how well is this covered in the rest of science? How well is the null-hypothesis covered? (I should say that I'm not trying to suggest that physics is as sloppy as psychology sometimes appears to be.) (EDIT: spelling)
- mathattack 12y ago
  It's a problem everywhere. I think Physics is somewhat spared because the gap between observation and math is less. (String theory perhaps being an exception) I think biochemistry runs into these issues more because it is harder to "prove" things with math. In general the less you can rely on supporting math or theory, the more you need to be careful about replicating experiments to come up with the truth.
  - cyorir 12y ago
    I think this is misunderstanding the role of math in science. Math is not used to "prove" scientific results, per se. Math is used to construct models, and prove that certain results would be expected from a given model. That doesn't mean the models work in the real world; all sciences use experiments to test models to see if they describe actual observations. In this sense, you always need to be careful about replication, regardless of field. If you can't show replication, then there is no guarantee that the model will always match observations, no matter how much math went into the construction of that model. In Physics the main problem is isolating variables, and setting up appropriate conditions. The good thing about Physics is that once the environment is set up and the experiment is well designed, it is easy to re-run the experiment for more trials. With chemistry and biology, this is usually the case as long as you have clean reagents in a clean lab, but you are introducing more factors of what you have to control as the complexity of your system scales up. It is hardest to replicate results involving humans, because with humans you have a lot of complexity, so you know you can never control all influencing factors. Usually in the social sciences, it is easier to hope these factors cancel out with a large, non-homogenous sample size. The problem as these experiments like SPE demonstrate is that it is hard to build an ideal sample and, if the ethics or practicality of your methods are questioned, rerun the experiment to replicate results.

This issue is true with much of social science. It would be great to see more replication (or non-replication) experiments to confirm what we think we do or don't know.

DanBC 12y ago

The linked comic is true - there are many problems with social science studies. But, since you specifically mention replication and non-replication -- how well is this covered in the rest of science? How well is the null-hypothesis covered? (I should say that I'm not trying to suggest that physics is as sloppy as psychology sometimes appears to be.) (EDIT: spelling)

mathattack 12y ago
It's a problem everywhere. I think Physics is somewhat spared because the gap between observation and math is less. (String theory perhaps being an exception) I think biochemistry runs into these issues more because it is harder to "prove" things with math. In general the less you can rely on supporting math or theory, the more you need to be careful about replicating experiments to come up with the truth.
- cyorir 12y ago
  I think this is misunderstanding the role of math in science. Math is not used to "prove" scientific results, per se. Math is used to construct models, and prove that certain results would be expected from a given model. That doesn't mean the models work in the real world; all sciences use experiments to test models to see if they describe actual observations. In this sense, you always need to be careful about replication, regardless of field. If you can't show replication, then there is no guarantee that the model will always match observations, no matter how much math went into the construction of that model. In Physics the main problem is isolating variables, and setting up appropriate conditions. The good thing about Physics is that once the environment is set up and the experiment is well designed, it is easy to re-run the experiment for more trials. With chemistry and biology, this is usually the case as long as you have clean reagents in a clean lab, but you are introducing more factors of what you have to control as the complexity of your system scales up. It is hardest to replicate results involving humans, because with humans you have a lot of complexity, so you know you can never control all influencing factors. Usually in the social sciences, it is easier to hope these factors cancel out with a large, non-homogenous sample size. The problem as these experiments like SPE demonstrate is that it is hard to build an ideal sample and, if the ethics or practicality of your methods are questioned, rerun the experiment to replicate results.