5 ms·

TIL. This is something new I learned. Can you tell me what to google to learn about this bias?

waterheater 5y ago
https://en.wikipedia.org/wiki/Statistical_significance https://en.wikipedia.org/wiki/Statistical_significance When the Higgs boson was declared "discovered," the particle was found to exist with a five-sigma certainty. This brief article does a great job of explaining what that means: https://www.r-bloggers.com/2012/07/the-higgs-boson-5-sigma-and-the-concept-of-p-values/ https://www.r-bloggers.com/2012/07/the-higgs-boson-5-sigma-a... Physicists got to that point (very roughly) by constructing complex mathematical models and building large machines to test those models. In contrast, leading psychology models change with the seasons, and large-scale human experiments cannot be (ethically) conducted with the control as seen in physics. Because psychology experiments have substantial environmental variability, drawing robust conclusions in psychology is exceedingly difficult. This fact should mean that very-high-certainty findings in psychology studies must be subjected to the highest scrutiny.
- CRConrad 5y ago
  I.e, far from being "2.9013", the question is whether this (bogus?) factor should be rounded to 3, or perhaps better "between 1 and 10".
  - klabb3 5y ago
    I think the implicit idea they wanted to sell was that a certain amount of positive thoughts creates a self-reinforcing psychological condition - making the grass greener in an endless cycle. You'd think that if this was true, religions, spiritual seekers and even random people would have discovered it independently many times over, and woven it into our collective and intuitive knowledge for thousands of years. But then again, the main figure behind this theory was a management consultant.
- anyfactor 5y ago
  Even though it is after 2 days, I came back to this comment. I was reading Weapons of Math Destruction and it just dawned on me. Tbh when I first read your comment I didn't understand it but know I do. In the book the author talks about baseball stats while you gave an example about physics.
nerdponx 5y ago
The more noise in the data, the less precisely you should claim to be able to measure it. Social science data is notoriously noisy and erratic. Relationships are notoriously weak. If your numbers have a lot of precision, more often than not it's just meaningless noise not "real" precision in measurement.

https://en.wikipedia.org/wiki/Statistical_significance https://en.wikipedia.org/wiki/Statistical_significance When the Higgs boson was declared "discovered," the particle was found to exist with a five-sigma certainty. This brief article does a great job of explaining what that means: https://www.r-bloggers.com/2012/07/the-higgs-boson-5-sigma-and-the-concept-of-p-values/ https://www.r-bloggers.com/2012/07/the-higgs-boson-5-sigma-a... Physicists got to that point (very roughly) by constructing complex mathematical models and building large machines to test those models. In contrast, leading psychology models change with the seasons, and large-scale human experiments cannot be (ethically) conducted with the control as seen in physics. Because psychology experiments have substantial environmental variability, drawing robust conclusions in psychology is exceedingly difficult. This fact should mean that very-high-certainty findings in psychology studies must be subjected to the highest scrutiny.

CRConrad 5y ago
I.e, far from being "2.9013", the question is whether this (bogus?) factor should be rounded to 3, or perhaps better "between 1 and 10".
- klabb3 5y ago
  I think the implicit idea they wanted to sell was that a certain amount of positive thoughts creates a self-reinforcing psychological condition - making the grass greener in an endless cycle. You'd think that if this was true, religions, spiritual seekers and even random people would have discovered it independently many times over, and woven it into our collective and intuitive knowledge for thousands of years. But then again, the main figure behind this theory was a management consultant.
anyfactor 5y ago
Even though it is after 2 days, I came back to this comment. I was reading Weapons of Math Destruction and it just dawned on me. Tbh when I first read your comment I didn't understand it but know I do. In the book the author talks about baseball stats while you gave an example about physics.

nerdponx 5y ago

The more noise in the data, the less precisely you should claim to be able to measure it. Social science data is notoriously noisy and erratic. Relationships are notoriously weak. If your numbers have a lot of precision, more often than not it's just meaningless noise not "real" precision in measurement.