3 ms·
> Warning Sign I5: Taking p too Seriously What does it say about X when you have 10 studies saying X is true with p < .05, and another 10 studies saying that X
by Alex3917 13y ago
> Warning Sign I5: Taking p too Seriously
What does it say about X when you have 10 studies saying X is true with p < .05, and another 10 studies saying that X is false with p < .05? It seems like the closer you come to actually taking p values at face value given conflicting results, the amount of things that can be considered to be 'scientifically proven' rapidly approaches zero.
Also, while I do like norvig's essay, all the errors he goes through are fairly pedestrian. If you read the modern scientific literature on research, the rabbit hole goes vastly deeper. (Assuming researchers researching researchers are more trustworthy than researchers.)
- einhverfr 13y ago> If you read the modern scientific literature on research, the rabbit hole goes vastly deeper. On indeed. Heisenberg used to say, "data does not imply theory."[1] Once you deeply internalize that, you realize you can't really be that, ahem, certain about anything in scientific thought outside of the limited domain of how scientific truth is defined (namely that the model has predictive value, which is entirely different from whether it has ontological value). [1] http://archive.org/details/PhysicsPhilosophy http://archive.org/details/PhysicsPhilosophy
- skylan_q 13y agoTry telling that to economists who claim to discover macroeconomic theories through econometric models and call it "empirical"
- dekhn 13y agoNothing is ever "proven". We merely make statements that become increasingly challenging to show are likely to be false based on human judgement of experimental methods. P values are estimates of the probability that a given observation is likely to have come about by chance, rather than due to a hypothetical model. They have many problems and it is best to think about them with a solid understanding of probability. If there was a case where you have as many conflicting results as you say, then it would be indicative of a very active scientific community and problem. People would be working hard to resolve the problem!