4 ms·

What about the rest of academic psychology for that matter? Is this endemic to that field?

walleeee 6y ago
The field is over-crowded, for one. Two, its participant pool is almost exclusively undergraduate students on university campuses in WEIRD (western, educated, industrialized, rich, and democratic) nations. Three, experimental designs may rely on questionably reliable, possibly retrospective self-report to infer patterns of affect and cognition. Four, experiments are often conducted in highly contrived laboratory environments. (In-situ data collection as people go about their lives is becoming more common with the rise of smartphones and wearables, however.) As in most domains of science, researchers are also pressured to publish regardless of the quality of the work, and replicated findings are usually not considered interesting or valuable enough to publish. This is anecdata, so take it with a grain of salt. (I minored in psychology and worked as a research assistant with a social/cognitive psych lab as an undergrad.) Also, there is a ton of extremely good science in psychology. It's a shame the recent profusion of low-quality work obscures it.
- lldbg 6y ago
  What is an example of extremely good science in psychology? What's a result we can hold up to scrutiny and believe truthfully with conviction?
  - shatnersbassoon 6y ago
    Cognitive psychology topics such as memory and reading have a strong paradigm, and established results. In general, their effects are much easier to replicate because you require far fewer participants. This is because you can validly make multiple observations of the same participant.
  - walleeee 6y ago
    As the other commenter mentioned, cognitive psychology is well-developed (working memory, the relationship between action impulses and our subjective awareness thereof, etc). And there is very good social psychological research on a number of useful topics including diffusion of responsibility, the fundamental attribution error, the effect of dissenting voices on groupthink, the influence of perceived authority on human decision-making (Milgram), etc. Kahneman and Tversky's contributions (and all of "behavioral economics") could also be considered social psychology. Anyways, though, I'm not sure it's super useful to "believe with conviction" in science. Shouldn't we hold all results up to scrutiny, and weigh them on the evidence? Along these lines, by far the most important takeaway from the last 50 years of academic psychology, imho, is that we are usually far too willing to trust our own personal judgment. The brain is as much a self-deception engine as it is a reasoning machine. We have a far hazier view of what actually goes on in our minds than we usually think we do.
shatnersbassoon 6y ago
I decided not to pursue an academic career after completing a Ph.D in Experimental Psychology. My main issue with it was: 1. You shouldn't know a priori whether you will be able to reject your null hypothesis (otherwise what is the point of doing the experiment). So what you need is luck, luck that your hunch turned out to be true. 2. If careers live and die by published results, then those who are lucky with finding significant effects early on in their career will win out. 3. Running a well-controlled experiment at scale is difficult in a way that I haven't found matched in the tech fields I have been in since leaving academia. I mean mega-hassle difficult. 4. You therefore have an incredible amount riding on the outcome of that experiment, because of its enormous opportunity cost. 5. The likelihood of being caught faking data is low (especially if you are halfway-competent which this researcher clearly is not). 6. The penalties of being caught faking data (as set out in this article) are relatively low. 7. The payoff of getting away with faking data can include a lot, up to and including a high profile academic career and tenure. 8. So from a game theoretic perspective, it's almost inevitable that quite a few people at the top have faked their way there. This is not to say that good work is not being done - there is some amazing work out there. I just think that like athletes taking steroids before they reach the big leagues, many academics succumbed to temptation to get an edge in order to get to the top. See the Dutch Social Psychology scandal for more on this.

The field is over-crowded, for one. Two, its participant pool is almost exclusively undergraduate students on university campuses in WEIRD (western, educated, industrialized, rich, and democratic) nations. Three, experimental designs may rely on questionably reliable, possibly retrospective self-report to infer patterns of affect and cognition. Four, experiments are often conducted in highly contrived laboratory environments. (In-situ data collection as people go about their lives is becoming more common with the rise of smartphones and wearables, however.) As in most domains of science, researchers are also pressured to publish regardless of the quality of the work, and replicated findings are usually not considered interesting or valuable enough to publish. This is anecdata, so take it with a grain of salt. (I minored in psychology and worked as a research assistant with a social/cognitive psych lab as an undergrad.) Also, there is a ton of extremely good science in psychology. It's a shame the recent profusion of low-quality work obscures it.

lldbg 6y ago
What is an example of extremely good science in psychology? What's a result we can hold up to scrutiny and believe truthfully with conviction?
- shatnersbassoon 6y ago
  Cognitive psychology topics such as memory and reading have a strong paradigm, and established results. In general, their effects are much easier to replicate because you require far fewer participants. This is because you can validly make multiple observations of the same participant.
- walleeee 6y ago
  As the other commenter mentioned, cognitive psychology is well-developed (working memory, the relationship between action impulses and our subjective awareness thereof, etc). And there is very good social psychological research on a number of useful topics including diffusion of responsibility, the fundamental attribution error, the effect of dissenting voices on groupthink, the influence of perceived authority on human decision-making (Milgram), etc. Kahneman and Tversky's contributions (and all of "behavioral economics") could also be considered social psychology. Anyways, though, I'm not sure it's super useful to "believe with conviction" in science. Shouldn't we hold all results up to scrutiny, and weigh them on the evidence? Along these lines, by far the most important takeaway from the last 50 years of academic psychology, imho, is that we are usually far too willing to trust our own personal judgment. The brain is as much a self-deception engine as it is a reasoning machine. We have a far hazier view of what actually goes on in our minds than we usually think we do.

shatnersbassoon 6y ago

I decided not to pursue an academic career after completing a Ph.D in Experimental Psychology. My main issue with it was: 1. You shouldn't know a priori whether you will be able to reject your null hypothesis (otherwise what is the point of doing the experiment). So what you need is luck, luck that your hunch turned out to be true. 2. If careers live and die by published results, then those who are lucky with finding significant effects early on in their career will win out. 3. Running a well-controlled experiment at scale is difficult in a way that I haven't found matched in the tech fields I have been in since leaving academia. I mean mega-hassle difficult. 4. You therefore have an incredible amount riding on the outcome of that experiment, because of its enormous opportunity cost. 5. The likelihood of being caught faking data is low (especially if you are halfway-competent which this researcher clearly is not). 6. The penalties of being caught faking data (as set out in this article) are relatively low. 7. The payoff of getting away with faking data can include a lot, up to and including a high profile academic career and tenure. 8. So from a game theoretic perspective, it's almost inevitable that quite a few people at the top have faked their way there. This is not to say that good work is not being done - there is some amazing work out there. I just think that like athletes taking steroids before they reach the big leagues, many academics succumbed to temptation to get an edge in order to get to the top. See the Dutch Social Psychology scandal for more on this.