3 ms·

The line between an invalid result and one that looks like it doesn't belong is fuzzy. I believe I was removing the former, but if an independent replicator doe

The line between an invalid result and one that looks like it doesn't belong is fuzzy. I believe I was removing the former, but if an independent replicator doesn't have access to the raw data they would never discover that I had been fooling myself. As to secrecy, there is nothing being hidden, instead it's that nothing in the system requires you to publish all your data, and so because you're pressed for time you don't. Never ascribe to malice what laziness can explain :)

endersshadow 15y ago
I'm not ascribing this to malice. It strikes me as a paragon of negligence if that's the way that the science of psychology is being done. In fact, it strikes me as an affront to science. Take this in contrast to the neutrinos that were going faster than light at CERN. Instead of going, "Okay, well, those can't happen, so those are outliers," they reported their results, published their methods and data, and said to the world, "Help us validate this." It seems to me like the modern psychology scientific method is the antithesis of this--"These are the results of my massaged data, and no you can't see how I got them, just trust me." I'm not attributing this to malice, I'm attributing this to negligence and laziness. Putting your work in front of people for critique is hard. Really hard. But, it's part of science. If psychology as a field has convinced themselves that they're above that reproach, then I think that's a huge condemnation of the field. When I first read the article, I thought it was a bit sensationalist to call psychology "under attack," but now I'm not so sure. I've assumed that reproducability is the standard litmus test of all science, but I guess not, and that leaves me with a bitter taste in my mouth toward the field of psychology.
- jmilloy 15y ago
  Outlier detection is an important part of basic statistics, and has been for a long time. It isn't about just deleting a few data points that "look" strange. Some statistical tests are robust to outliers and some are not... it's always important to use the appropriate tests in the appropriate way and be open about them.

endersshadow 15y ago

I'm not ascribing this to malice. It strikes me as a paragon of negligence if that's the way that the science of psychology is being done. In fact, it strikes me as an affront to science. Take this in contrast to the neutrinos that were going faster than light at CERN. Instead of going, "Okay, well, those can't happen, so those are outliers," they reported their results, published their methods and data, and said to the world, "Help us validate this." It seems to me like the modern psychology scientific method is the antithesis of this--"These are the results of my massaged data, and no you can't see how I got them, just trust me." I'm not attributing this to malice, I'm attributing this to negligence and laziness. Putting your work in front of people for critique is hard. Really hard. But, it's part of science. If psychology as a field has convinced themselves that they're above that reproach, then I think that's a huge condemnation of the field. When I first read the article, I thought it was a bit sensationalist to call psychology "under attack," but now I'm not so sure. I've assumed that reproducability is the standard litmus test of all science, but I guess not, and that leaves me with a bitter taste in my mouth toward the field of psychology.

jmilloy 15y ago
Outlier detection is an important part of basic statistics, and has been for a long time. It isn't about just deleting a few data points that "look" strange. Some statistical tests are robust to outliers and some are not... it's always important to use the appropriate tests in the appropriate way and be open about them.