3 ms·

I'm not an expert on experimental psychology, but I'm fairly certain that the principles of how experiments are conducted in the sense that they control for oth

I'm not an expert on experimental psychology, but I'm fairly certain that the principles of how experiments are conducted in the sense that they control for other factors, choose sufficiently large samples etc., are pretty sound. An often mentioned problem in psychology research is "p-hacking", where researchers manipulate the data in a way that validates their hypotheses statistically, which seems like a more logical reason for a reproducibility crisis than the reasons you mentioned. Also just because there are some problems in the field should not cause you to dismiss science completely in favor of completely subjective speculating about how the mind works, which certainly also has value, but I don't think it's warranted to dismiss actual science in favor of it.

stanfordkid 8y ago
But they're just not -- that's exactly my point. If you test a bunch of Harvard undergrads (the most common subject type in these studies) -- that's not a uniform sample. In any way whatsoever. Furthermore, even if you get a high p-value ... your experimental design can introduce so much noise. Even when you do find these high p-value correlations -- they don't really ever say much about "why" ... because they can't. So much of that is the study design. Let's say I make a study on how well people navigate a maze under the influence of alcohol vs. not. You can probably get a high p-value that they do worse while under the influence of alcohol... but it reveals very little about why. So much of that is dependent on the particular maze. In fact I bet I could design a maze specifically to prove whatever conclusion I wanted. That is the whole problem in psychology. There is very little focus given to actually characterizing cognition in any meaningful way. I like the clinical psychologists because they attempt to do exactly that, even though they have less "scientific" findings.

stanfordkid 8y ago

But they're just not -- that's exactly my point. If you test a bunch of Harvard undergrads (the most common subject type in these studies) -- that's not a uniform sample. In any way whatsoever. Furthermore, even if you get a high p-value ... your experimental design can introduce so much noise. Even when you do find these high p-value correlations -- they don't really ever say much about "why" ... because they can't. So much of that is the study design. Let's say I make a study on how well people navigate a maze under the influence of alcohol vs. not. You can probably get a high p-value that they do worse while under the influence of alcohol... but it reveals very little about why. So much of that is dependent on the particular maze. In fact I bet I could design a maze specifically to prove whatever conclusion I wanted. That is the whole problem in psychology. There is very little focus given to actually characterizing cognition in any meaningful way. I like the clinical psychologists because they attempt to do exactly that, even though they have less "scientific" findings.