3 ms·

> confirmable But were they indeed confirmed? This is the problem that keeps cropping up. Sometimes I wonder if believing in findings from the psychological

by apathy 10y ago

> confirmable But were they indeed confirmed? This is the problem that keeps cropping up. Sometimes I wonder if believing in findings from the psychological literature isn't a disadvantage. Knowing about them is good -- you can pick up on the chumps who actually believe them. But believing them? It's not at all clear to me that uncritical acceptance of these studies is ever a good idea. The response of senior investigators shrilly decrying "methodological terrorism" when their small-sample findings fail to replicate does little to increase confidence in said findings (or, indeed, the field as a whole). nb. In no way is this confined to psychology. It's just more rampant there, to judge by the foundational manner in which the field has been shaken up. Nobody doubts that PCR works or that iPSCs can be induced, because the experiments to replicate the results are relatively trivial. But the nature of psychological research seems to make such exercises very difficult, to the point that individuals "take it on faith" that an effect is real (as opposed to assuming it's wrong and running a quick experiment to show that the next year or two of their life won't be wasted). Maybe we're just more cynical in bench-based disciplines?

artifaxx 10y ago
Psychology definitely needs a higher level of rigor in their studies. It is frustrating but you just have to pick apart the methodology more rigorously to be reasonably certain. Understanding how our brains work even at a rudimentary level is too useful to pass up.
unoti 10y ago
> But were they indeed confirmed? This is the problem that keeps cropping up. The book is carefully researched and exquisitely footnoted. Can you cite something specific that you have a problem with in the book? I'm curious if you have even read this book or if you just stating a more general concern about people who do me publish research.
- apathy 10y ago
  I can't quite understand your last sentence -- I publish, as do most academics, so that can't be it. But Cialdini is the "pre-suasion" guy, and that one has certainly failed to replicate. So I guess I have a prior skepticism about anything from a standard textbook publisher covering similar topic. I've found too many errors in exercises, etc. to believe that their editors are doing much of a job; it's a drag when you're teaching a stats methods class and the book questions have major numeric errors so you can't use it for homework. What's the purpose of the book at that point? Next time I teach (actually the past few as well) I'm not going to bother with a textbook. Go pull these primary research articles, pull the data, and see if the methods were used correctly. The end. As previously, I've had some surprised looks when students did this with biological data sets. It's not like psychology is some rogue's gallery; it's a widespread problem. Only solution is to verify results before building upon them.

by apathy 10y ago

artifaxx 10y ago

Psychology definitely needs a higher level of rigor in their studies. It is frustrating but you just have to pick apart the methodology more rigorously to be reasonably certain. Understanding how our brains work even at a rudimentary level is too useful to pass up.

unoti 10y ago

> But were they indeed confirmed? This is the problem that keeps cropping up. The book is carefully researched and exquisitely footnoted. Can you cite something specific that you have a problem with in the book? I'm curious if you have even read this book or if you just stating a more general concern about people who do me publish research.

apathy 10y ago
I can't quite understand your last sentence -- I publish, as do most academics, so that can't be it. But Cialdini is the "pre-suasion" guy, and that one has certainly failed to replicate. So I guess I have a prior skepticism about anything from a standard textbook publisher covering similar topic. I've found too many errors in exercises, etc. to believe that their editors are doing much of a job; it's a drag when you're teaching a stats methods class and the book questions have major numeric errors so you can't use it for homework. What's the purpose of the book at that point? Next time I teach (actually the past few as well) I'm not going to bother with a textbook. Go pull these primary research articles, pull the data, and see if the methods were used correctly. The end. As previously, I've had some surprised looks when students did this with biological data sets. It's not like psychology is some rogue's gallery; it's a widespread problem. Only solution is to verify results before building upon them.