7 ms·

I would take it even farther and claim that very little in experimental psychology / sociology has any value whatsoever. Google the top papers of all time in ps

I would take it even farther and claim that very little in experimental psychology / sociology has any value whatsoever. Google the top papers of all time in psychology and its either been formally debunked (Stanford prison experiment) or it has no applicable impact on society. Using computers to parse statistical data is interesting and can reveal truths about society. Trying to correlate the delay between eating a marshmellow as a child and income as an adult is ridiculous, yet collectively we have spent untold money and time on tenured professors, conferences, papers, and pop psychology books about such things. Another example is Power Posing. How Amy Cuddy still gets employment is a mystery.

spekcular 4y ago
The marshmallow effect replicated (IIRC), and is probably correct, unlike the power posing stuff. That behavior in a single trial in childhood moderately predicts success in adulthood seems like an interesting fact about humans.
- monero-xmr 4y ago
  What's the applicable result to this? Put children who eat marshmellows too quickly in special "delayed gratification" classes? Is the result such that all children who eat marshmellows too quickly should give up, as their success is predetermined, or is it the impact is barely statistically significant? If I were the parent of a child who ate the marshmellow too quickly, I would do nothing different, as the whole thing is meaningless and doesn't change anything. It's more like, "Oh, OK, my son is 4% less likely to be in the top 20% of the income distribution (according to this barely coherent theory)." Why do we spend so much money, time, and effort discussing and researching this?
  - hgsgm 4y ago
    What? If you found your child was lacking in a valuable life skill, you wouldn't try to teach that skill?
    - nonethewiser 4y ago
      But that's precisely what is so ridiculous. Eating marshmallows slowly is not a valuable life skill.
    - teruakohatu 4y ago
      What is the skill they should teach ? To always eat lunch before doing a psych experiment? To like sugar, or marshmallows in particular, just slightly less? That high self-control in extremely low-reward situations is good? Teaching a child to do basic budgeting with pocket money is likely to have a vastly higher impact than slight delayed gratification with candy.
    - BCM43 4y ago
      See hgsgm's comment. Just because the test predicts outcomes doesn't mean what it's testing is causing the outcomes.
      - Izkata 4y ago
        Uh, same person.
  - mplanchard 4y ago
    Lots of research doesn’t have an immediate practical application. That’s okay. It can still be interesting, and it may prove practical down the road.
  - spekcular 4y ago
    The experiment is part of a larger research program to study delayed gratification. It was never intended to help parents raise their children. Asking for immediate applications is an odd standard for basic science research. (Unless you reject the value of basic science research entirely.) Further, the result in a recent replication [1] was a correlation of .28 between the time to ring the bell (to get the marshmallow) and academic achievement. That's not exactly the trivial "4% less likely" effect in your caricature. (You may also wish to double check your spelling of "marshmallow.") [1]: https://www.jasoncollins.blog/posts/the-marshmallow-test-held-up-ok https://www.jasoncollins.blog/posts/the-marshmallow-test-hel...
    - ogogmad 4y ago
      > correlation of .28 ... That's not exactly the trivial "4% less likely" effect in your caricature. You clearly don't know how to interpret correlation. A correlation of .28 is virtually uninformative. https://www.academia.edu/39797871/Fooled_by_Correlation_Common_Misinterpretations_in_Social_Science_ https://www.academia.edu/39797871/Fooled_by_Correlation_Comm...
- Ygg2 4y ago
  Sure it's replicated. It disappears when you control for affluence of parents. I.e. it's a covert test if parents are rich.
  - kneebonian 4y ago
    I know it's super taboo to consider especially in the current climate but is it possible that people who have high self control are more likely to become rich because of their ability to delay gratification and that therefore those genes will be passed on to their children?
    - pvaldes 4y ago
      Biology will select against this. All primate females seem to be much more attracted by bold males taking risks, and more prone to use sex to appease dominant males showing violent episodes. The opposite argument could be equally valid. The only thing that children of rich parents share is to have rich parents. One of the parents being over the mean in the "beautiful" trait, is also very common.
  - mvnuweucxqokii 4y ago
    If I understand correctly, you're asserting that that's causal--that being born into affluence causes children to develop in a way that leads to them "passing" the marshmallow test. Do you have evidence to exclude other possible interpretations, such as the possibility that children resisting the temptation of a marshmallow and their parents being rich are actually the effects of some other underlying single cause?
    - Ygg2 4y ago
      > If I understand correctly, you're asserting that that's causal--that being born into affluence causes children to develop in a way that leads to them "passing" the marshmallow test. Stop and think. If you are rich that marshmallow is probably very bland or you've eaten very well today. So maybe two makes more sense. If you aren't, that marshmallow is super tasty and you probably didn't eat well today. Plus do you believe that grownup when your mom was lying. Someone might come and eat it. Then some time passes. Rich kids grow up and are more successful. Delayed gratification is key to success! https://behavioralscientist.org/try-to-resist-misinterpreting-the-marshmallow-test/ https://behavioralscientist.org/try-to-resist-misinterpretin...
dataflow 4y ago
What about all the research showing how easy it is to form false memories? Surely that's applicable to society?
- mistermann 4y ago
  Propaganda certainly seems to work quite well, perhaps it would work less well if (the good parts of) psychology was a mandatory part of basic curriculum.
hgsgm 4y ago
OP has written extensively about Power Pose. Marshmallow effect is real, but has important confounders like "stable home life with trustwor5 adults". https://statmodeling.stat.columbia.edu/2017/10/18/beyond-power-pose-using-replication-failures-better-understanding-data-collection-analysis-better-science/ https://statmodeling.stat.columbia.edu/2017/10/18/beyond-pow...
thereticent 4y ago
Take what you said and apply the same logic to privacy-focused cryptocurrencies. You are armchair quarterbacking and speaking beyond your expertise, which is fine, but the claim you make is too strong for the supposed evidence you mention. We'd probably all agree that's worth keeping an eye on your own credulity. That applies both to believing the scientists themselves and to believing critics that the science is overall bunk.
- mannerheim 4y ago
  > That applies both to believing the scientists themselves I believe Feynman when it comes to psychology.
  - mistermann 4y ago
    Are you sure? "'I think the educational and psychological studies I mentioned..." This is not a wholesale dismissal of the entire domain. That so many people interpret it that way is a psychological phenomenon in itself.
    - mannerheim 4y ago
      'Other kinds of errors are more characteristic of poor science. When I was at Cornell. I often talked to the people in the psychology department. One of the students told me she wanted to do an experiment that went something like this—I don’t remember it in detail, but it had been found by others that under certain circumstances, X, rats did something, A. She was curious as to whether, if she changed the circumstances to Y, they would still do, A. So her proposal was to do the experiment under circumstances Y and see if they still did A. I explained to her that it was necessary first to repeat in her laboratory the experiment of the other person—to do it under condition X to see if she could also get result A—and then change to Y and see if A changed. Then she would know that the real difference was the thing she thought she had under control. She was very delighted with this new idea, and went to her professor. And his reply was, no, you cannot do that, because the experiment has already been done and you would be wasting time. This was in about 1935 or so, and it seems to have been the general policy then to not try to repeat psychological experiments, but only to change the conditions and see what happens.' Judging by the replication crisis in psychology, not much has changed since Feynman's day.
skissane 4y ago
Here is one of my favourite papers: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6880188/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6880188/ - it is published in a psychiatry journal not a psychology journal, but it is absolutely an area which is studied in psychology-there is no clearcut boundary between psychology research and psychiatry research anyway The paper adds to the growing body of evidence that DSM-5 diagnoses lack validity. The study specifically looks at ASD, ADHD and OCD in children/adolescents, but there are other studies similarly examining other diagnoses. I think this is very important research, because if we are going to use psychiatric diagnoses, we have to support research which questions the evidence behind them. If we don’t think that kind of research is worthwhile, then either we stop using them altogether, or else we are turning them into something faith-based rather than science-based.
- nradov 4y ago
  I don't think anyone in the field seriously claims that DSM-5 diagnoses have some particular validity. Most of them are just clusters of highly correlated symptoms without a clear, objective causal link to any underlying pathology. But as a practical matter, in clinical practice it's useful to have labels for those clusters of symptoms. This helps to facilitate communications between providers, as well as in generating the documentation required by payers to justify insurance claims.
  - skissane 4y ago
    > I don't think anyone in the field seriously claims that DSM-5 diagnoses have some particular validity In my personal experience, many clinicians – and even researchers – talk about diagnoses in a reified way which only makes sense if one presumes they do have some sort of validity. Many of them also appear to engage in the "motte-and-bailey fallacy" – talking about diagnoses as if they were valid, falling back on "we all know they aren't really" whenever that talk is challenged, and then going straight back to talking that way as soon as the challenge has departed. If one takes the lack of validity of DSM-5 diagnoses seriously, that has certain consequences for research programming and study design – that one ought to prefer trans-diagnostic studies to those focusing on single diagnoses, that one ought to study cohorts which include subclinical cases along with those with a clinical diagnosis, etc – and yet, very many researchers continue to ignore all those recommendations, which suggests they don't actually take that lack of validity seriously at all. In 2017, the journal Autism Research published a letter to the editors, by Lynn Waterhouse, Eric London and Christopher Gillberg, entitled "The ASD diagnosis has blocked the discovery of valid biological variation in neurodevelopmental social impairment" [0] – the authors argue that focusing research on a diagnosis which lacks validity is a scientific dead-end, that it is no wonder that it has failed to produce the results which its promoters had promised, and that the field is going to continue to be fruitless until research is re-oriented away from its current focus on an invalid diagnostic category. And I'm sure they'd say that, while their letter was about ASD specifically, the same is likely true for many other DSM-5 diagnoses as well > Most of them are just clusters of highly correlated symptoms without a clear, objective causal link to any underlying pathology. How strong is the evidence that these symptoms actually are "highly correlated"? To take ASD as an example, the diagnostic criteria refer to problems in two separate domains – social communication, and restricted/repetitive behaviours/interests (RRBIs) – with problems in both domains required for a positive diagnosis. This linkage is justified on the grounds that the two symptom domains are "highly correlated". However, the scientific evidence for the strength of that claimed correlation is actually rather weak and lacking – see [1]. And if that is true for ASD, it is probably true for many other diagnoses as well > This helps to facilitate communications between providers If the point was really to facilitate communication between providers, a better approach would be to just provide a list of symptoms for each patient/client. DSM-5 diagnoses are less useful than that, because many of them are defined in terms such as "Exhibits at least M of the following N:", so two people can both have the same diagnosis without actually having any symptoms in common. > as well as in generating the documentation required by payers to justify insurance claims. Well, here we get to the real point of diagnosis – a collection of cultural constructs in which policy-makers have faith, so they'll construct policies (insurance, disability service funding, research funding, drug approval, etc) oriented around those cultural constructs – and they display a studied ignorance of the lack of scientific evidence to support what they are doing, and also the very real possibility that doing things that way is at best useless and at worst positively harmful (see [2]) [0] https://pubmed.ncbi.nlm.nih.gov/28714261/ https://pubmed.ncbi.nlm.nih.gov/28714261/ [1] https://pubmed.ncbi.nlm.nih.gov/18564070/ https://pubmed.ncbi.nlm.nih.gov/18564070/ [2] https://pubmed.ncbi.nlm.nih.gov/22250827/ https://pubmed.ncbi.nlm.nih.gov/22250827/
danaris 4y ago
The really big, splashy papers like the Stanford Prison Experiment and Milgram are partly well-known because of what bad science they are. They're also 50 years old. If that's your idea of what most of psychology is like, then I imagine you think 90% physics is all about things like phlogiston. Just because there are some very well-known experiments that were very poorly conceived and conducted does not say anything about the field as a whole. As someone who works in tech for a psychology department, I can tell you from a decade's experience that a) the people there care very deeply about performing good science, including replicating results, pre-registering papers, and doing good statistics (not p-hacking), b) there's a lot of psychological research that's decidedly unsexy, very much about things like proving what we already intuitively know to be true or sussing out small nuances between X and Y, and c) while they teach about Milgram and his ilk, it's almost entirely in a "What Not To Do" type of way.
- mannerheim 4y ago
  SPE is still written about uncritically in psychology textbooks. If there are any criticisms, they're ethical criticisms rather than the fact that SPE was scientifically garbage. If physics textbooks still spoke uncritically about phlogiston theory, you'd have a point.
  - kuhewa 4y ago
    e.g.?
    - mannerheim 4y ago
      https://journals.sagepub.com/doi/10.1177/0098628314537968 https://journals.sagepub.com/doi/10.1177/0098628314537968
      - kuhewa 4y ago
        Thanks. Well, textbooks last copyrighted a decade ago, right as the reckoning in experimental psychology around the replication crisis was taking off. I suspect they may look different now.
ordu 4y ago
> I would take it even farther and claim that very little in experimental psychology / sociology has any value whatsoever. You are right, probably. I'm not exactly sure, because "top papers" like Standford Prison Experiment, are at the top for reasons that seems to me arbitrary and far away from a science. There are a lot of very good articles, some authors are really dig into their problems. Though these are not known for a public. They do not make good headlines. It is really annoying, because basically all popular psychology is a junk-science or have nothing to do with a science at all. But in any case we can agree that there are a lot of papers with little to no value. And as I see it is because psychology and sociology are on the quest of creating the Method. The Method of research that works in their discipline. They do better now, then it was before. Now a paper like Zimbardo's wouldn't stand a chance. Papers from 30 years ago probably wouldn't seem rigorous enough now. The Method is becoming better, and probably it will become even better in a future and today's research will be deemed not good enough to be science, and will be read by undergraduates as a cautionary tale how one mustn't do Science. It is a little like programming which changes with each passing decade, though a little slower, I'd guess. But socially it is different. I mean if we look not at results of research but on a social processes leading to this result, it is different from programming. Programmers are expected to create programs that work, they need to deliver. There is a final clear criterion of a success. Psychologists do not have such a criterion. They seek Truth, but what Truth is and what isn't is debatable. And they work for grants, not for revenue. Programmers are inventing new tools and techniques allowing to create better or more complex programs, but at the same time they also need to deliver something useful for non-programmers. Psychologists are inventing new tools and techniques of research (or sharpen old ones), but they do not feel the urge to deliver something to non-researchers. And to be honest oftentimes they are unable to deliver. They can find some grain of truth, but no one can find a practical application for this grain yet. I think it will get better when neuroscience, AI, and psychology converge and some kind of a psychological theory will emerge. They are doing it now, and they are bound to collide at some point in a future. There are lots of psychology theories now, but no physicist will agree that any of them deserves to be called a theory. They are more like vague descriptions than theories. Or they are talk about some small part of mind. I cannot say about prospects of sociology because I know little about it, but I'd say it would take more time to bring sociology to a high standards of an empirical science. > Trying to correlate the delay between eating a marshmellow as a child and income as an adult is ridiculous, yet collectively we have spent untold money and time on tenured professors, conferences, papers, and pop psychology books about such things. Another example is Power Posing. How Amy Cuddy still gets employment is a mystery. May be it is ridiculous. But if we want to know how human mind works, we better pay for it. Researchers played with electricity for centuries. Michael Faraday himself spent half a century designing experiments. All that didn't deliver nothing of a real practical value. But then James Clerk Maxwell published his equations. And suddenly electricity started to deliver. Electricity delivered probably more than any other technology.
spritefs 4y ago
> I would take it even farther and claim that very little in experimental psychology / sociology has any value whatsoever. Oh no they definitely have value, in the same way that metaphysics has value
mattkrause 4y ago
“Experimental psychology” might be a bigger field than you imagine. It includes tons of work characterizing sensory perception. These studies tend to replicate pretty well and they have a pretty broad range of real-world applications: psychoacoustic models for audio compression, principles like pop-out and serial/parallel search that inform user interfaces, color reproduction. In fact, psychophysical experiments pinned down the properties of cone cells over a hundred years before they were directly measured. Psychometrics, the study of test/assessment design, also shows up all over the place: education, obviously, but also marketing and other forms of evaluation.
melagonster 4y ago
the example of marshmellow was modified by original researcher. reversely, this is a example of good research.

spekcular 4y ago

The marshmallow effect replicated (IIRC), and is probably correct, unlike the power posing stuff. That behavior in a single trial in childhood moderately predicts success in adulthood seems like an interesting fact about humans.

monero-xmr 4y ago
What's the applicable result to this? Put children who eat marshmellows too quickly in special "delayed gratification" classes? Is the result such that all children who eat marshmellows too quickly should give up, as their success is predetermined, or is it the impact is barely statistically significant? If I were the parent of a child who ate the marshmellow too quickly, I would do nothing different, as the whole thing is meaningless and doesn't change anything. It's more like, "Oh, OK, my son is 4% less likely to be in the top 20% of the income distribution (according to this barely coherent theory)." Why do we spend so much money, time, and effort discussing and researching this?
- hgsgm 4y ago
  What? If you found your child was lacking in a valuable life skill, you wouldn't try to teach that skill?
  - nonethewiser 4y ago
    But that's precisely what is so ridiculous. Eating marshmallows slowly is not a valuable life skill.
  - teruakohatu 4y ago
    What is the skill they should teach ? To always eat lunch before doing a psych experiment? To like sugar, or marshmallows in particular, just slightly less? That high self-control in extremely low-reward situations is good? Teaching a child to do basic budgeting with pocket money is likely to have a vastly higher impact than slight delayed gratification with candy.
  - BCM43 4y ago
    See hgsgm's comment. Just because the test predicts outcomes doesn't mean what it's testing is causing the outcomes.
    - Izkata 4y ago
      Uh, same person.
- mplanchard 4y ago
  Lots of research doesn’t have an immediate practical application. That’s okay. It can still be interesting, and it may prove practical down the road.
- spekcular 4y ago
  The experiment is part of a larger research program to study delayed gratification. It was never intended to help parents raise their children. Asking for immediate applications is an odd standard for basic science research. (Unless you reject the value of basic science research entirely.) Further, the result in a recent replication [1] was a correlation of .28 between the time to ring the bell (to get the marshmallow) and academic achievement. That's not exactly the trivial "4% less likely" effect in your caricature. (You may also wish to double check your spelling of "marshmallow.") [1]: https://www.jasoncollins.blog/posts/the-marshmallow-test-held-up-ok https://www.jasoncollins.blog/posts/the-marshmallow-test-hel...
  - ogogmad 4y ago
    > correlation of .28 ... That's not exactly the trivial "4% less likely" effect in your caricature. You clearly don't know how to interpret correlation. A correlation of .28 is virtually uninformative. https://www.academia.edu/39797871/Fooled_by_Correlation_Common_Misinterpretations_in_Social_Science_ https://www.academia.edu/39797871/Fooled_by_Correlation_Comm...
Ygg2 4y ago
Sure it's replicated. It disappears when you control for affluence of parents. I.e. it's a covert test if parents are rich.
- kneebonian 4y ago
  I know it's super taboo to consider especially in the current climate but is it possible that people who have high self control are more likely to become rich because of their ability to delay gratification and that therefore those genes will be passed on to their children?
  - pvaldes 4y ago
    Biology will select against this. All primate females seem to be much more attracted by bold males taking risks, and more prone to use sex to appease dominant males showing violent episodes. The opposite argument could be equally valid. The only thing that children of rich parents share is to have rich parents. One of the parents being over the mean in the "beautiful" trait, is also very common.
- mvnuweucxqokii 4y ago
  If I understand correctly, you're asserting that that's causal--that being born into affluence causes children to develop in a way that leads to them "passing" the marshmallow test. Do you have evidence to exclude other possible interpretations, such as the possibility that children resisting the temptation of a marshmallow and their parents being rich are actually the effects of some other underlying single cause?
  - Ygg2 4y ago
    > If I understand correctly, you're asserting that that's causal--that being born into affluence causes children to develop in a way that leads to them "passing" the marshmallow test. Stop and think. If you are rich that marshmallow is probably very bland or you've eaten very well today. So maybe two makes more sense. If you aren't, that marshmallow is super tasty and you probably didn't eat well today. Plus do you believe that grownup when your mom was lying. Someone might come and eat it. Then some time passes. Rich kids grow up and are more successful. Delayed gratification is key to success! https://behavioralscientist.org/try-to-resist-misinterpreting-the-marshmallow-test/ https://behavioralscientist.org/try-to-resist-misinterpretin...

dataflow 4y ago

What about all the research showing how easy it is to form false memories? Surely that's applicable to society?

mistermann 4y ago
Propaganda certainly seems to work quite well, perhaps it would work less well if (the good parts of) psychology was a mandatory part of basic curriculum.

hgsgm 4y ago

OP has written extensively about Power Pose. Marshmallow effect is real, but has important confounders like "stable home life with trustwor5 adults". https://statmodeling.stat.columbia.edu/2017/10/18/beyond-power-pose-using-replication-failures-better-understanding-data-collection-analysis-better-science/ https://statmodeling.stat.columbia.edu/2017/10/18/beyond-pow...

thereticent 4y ago

Take what you said and apply the same logic to privacy-focused cryptocurrencies. You are armchair quarterbacking and speaking beyond your expertise, which is fine, but the claim you make is too strong for the supposed evidence you mention. We'd probably all agree that's worth keeping an eye on your own credulity. That applies both to believing the scientists themselves and to believing critics that the science is overall bunk.

mannerheim 4y ago
> That applies both to believing the scientists themselves I believe Feynman when it comes to psychology.
- mistermann 4y ago
  Are you sure? "'I think the educational and psychological studies I mentioned..." This is not a wholesale dismissal of the entire domain. That so many people interpret it that way is a psychological phenomenon in itself.
  - mannerheim 4y ago
    'Other kinds of errors are more characteristic of poor science. When I was at Cornell. I often talked to the people in the psychology department. One of the students told me she wanted to do an experiment that went something like this—I don’t remember it in detail, but it had been found by others that under certain circumstances, X, rats did something, A. She was curious as to whether, if she changed the circumstances to Y, they would still do, A. So her proposal was to do the experiment under circumstances Y and see if they still did A. I explained to her that it was necessary first to repeat in her laboratory the experiment of the other person—to do it under condition X to see if she could also get result A—and then change to Y and see if A changed. Then she would know that the real difference was the thing she thought she had under control. She was very delighted with this new idea, and went to her professor. And his reply was, no, you cannot do that, because the experiment has already been done and you would be wasting time. This was in about 1935 or so, and it seems to have been the general policy then to not try to repeat psychological experiments, but only to change the conditions and see what happens.' Judging by the replication crisis in psychology, not much has changed since Feynman's day.

skissane 4y ago

Here is one of my favourite papers: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6880188/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6880188/ - it is published in a psychiatry journal not a psychology journal, but it is absolutely an area which is studied in psychology-there is no clearcut boundary between psychology research and psychiatry research anyway The paper adds to the growing body of evidence that DSM-5 diagnoses lack validity. The study specifically looks at ASD, ADHD and OCD in children/adolescents, but there are other studies similarly examining other diagnoses. I think this is very important research, because if we are going to use psychiatric diagnoses, we have to support research which questions the evidence behind them. If we don’t think that kind of research is worthwhile, then either we stop using them altogether, or else we are turning them into something faith-based rather than science-based.

nradov 4y ago
I don't think anyone in the field seriously claims that DSM-5 diagnoses have some particular validity. Most of them are just clusters of highly correlated symptoms without a clear, objective causal link to any underlying pathology. But as a practical matter, in clinical practice it's useful to have labels for those clusters of symptoms. This helps to facilitate communications between providers, as well as in generating the documentation required by payers to justify insurance claims.
- skissane 4y ago
  > I don't think anyone in the field seriously claims that DSM-5 diagnoses have some particular validity In my personal experience, many clinicians – and even researchers – talk about diagnoses in a reified way which only makes sense if one presumes they do have some sort of validity. Many of them also appear to engage in the "motte-and-bailey fallacy" – talking about diagnoses as if they were valid, falling back on "we all know they aren't really" whenever that talk is challenged, and then going straight back to talking that way as soon as the challenge has departed. If one takes the lack of validity of DSM-5 diagnoses seriously, that has certain consequences for research programming and study design – that one ought to prefer trans-diagnostic studies to those focusing on single diagnoses, that one ought to study cohorts which include subclinical cases along with those with a clinical diagnosis, etc – and yet, very many researchers continue to ignore all those recommendations, which suggests they don't actually take that lack of validity seriously at all. In 2017, the journal Autism Research published a letter to the editors, by Lynn Waterhouse, Eric London and Christopher Gillberg, entitled "The ASD diagnosis has blocked the discovery of valid biological variation in neurodevelopmental social impairment" [0] – the authors argue that focusing research on a diagnosis which lacks validity is a scientific dead-end, that it is no wonder that it has failed to produce the results which its promoters had promised, and that the field is going to continue to be fruitless until research is re-oriented away from its current focus on an invalid diagnostic category. And I'm sure they'd say that, while their letter was about ASD specifically, the same is likely true for many other DSM-5 diagnoses as well > Most of them are just clusters of highly correlated symptoms without a clear, objective causal link to any underlying pathology. How strong is the evidence that these symptoms actually are "highly correlated"? To take ASD as an example, the diagnostic criteria refer to problems in two separate domains – social communication, and restricted/repetitive behaviours/interests (RRBIs) – with problems in both domains required for a positive diagnosis. This linkage is justified on the grounds that the two symptom domains are "highly correlated". However, the scientific evidence for the strength of that claimed correlation is actually rather weak and lacking – see [1]. And if that is true for ASD, it is probably true for many other diagnoses as well > This helps to facilitate communications between providers If the point was really to facilitate communication between providers, a better approach would be to just provide a list of symptoms for each patient/client. DSM-5 diagnoses are less useful than that, because many of them are defined in terms such as "Exhibits at least M of the following N:", so two people can both have the same diagnosis without actually having any symptoms in common. > as well as in generating the documentation required by payers to justify insurance claims. Well, here we get to the real point of diagnosis – a collection of cultural constructs in which policy-makers have faith, so they'll construct policies (insurance, disability service funding, research funding, drug approval, etc) oriented around those cultural constructs – and they display a studied ignorance of the lack of scientific evidence to support what they are doing, and also the very real possibility that doing things that way is at best useless and at worst positively harmful (see [2]) [0] https://pubmed.ncbi.nlm.nih.gov/28714261/ https://pubmed.ncbi.nlm.nih.gov/28714261/ [1] https://pubmed.ncbi.nlm.nih.gov/18564070/ https://pubmed.ncbi.nlm.nih.gov/18564070/ [2] https://pubmed.ncbi.nlm.nih.gov/22250827/ https://pubmed.ncbi.nlm.nih.gov/22250827/

danaris 4y ago

The really big, splashy papers like the Stanford Prison Experiment and Milgram are partly well-known because of what bad science they are. They're also 50 years old. If that's your idea of what most of psychology is like, then I imagine you think 90% physics is all about things like phlogiston. Just because there are some very well-known experiments that were very poorly conceived and conducted does not say anything about the field as a whole. As someone who works in tech for a psychology department, I can tell you from a decade's experience that a) the people there care very deeply about performing good science, including replicating results, pre-registering papers, and doing good statistics (not p-hacking), b) there's a lot of psychological research that's decidedly unsexy, very much about things like proving what we already intuitively know to be true or sussing out small nuances between X and Y, and c) while they teach about Milgram and his ilk, it's almost entirely in a "What Not To Do" type of way.

mannerheim 4y ago
SPE is still written about uncritically in psychology textbooks. If there are any criticisms, they're ethical criticisms rather than the fact that SPE was scientifically garbage. If physics textbooks still spoke uncritically about phlogiston theory, you'd have a point.
- kuhewa 4y ago
  e.g.?
  - mannerheim 4y ago
    https://journals.sagepub.com/doi/10.1177/0098628314537968 https://journals.sagepub.com/doi/10.1177/0098628314537968
    - kuhewa 4y ago
      Thanks. Well, textbooks last copyrighted a decade ago, right as the reckoning in experimental psychology around the replication crisis was taking off. I suspect they may look different now.

ordu 4y ago

> I would take it even farther and claim that very little in experimental psychology / sociology has any value whatsoever. You are right, probably. I'm not exactly sure, because "top papers" like Standford Prison Experiment, are at the top for reasons that seems to me arbitrary and far away from a science. There are a lot of very good articles, some authors are really dig into their problems. Though these are not known for a public. They do not make good headlines. It is really annoying, because basically all popular psychology is a junk-science or have nothing to do with a science at all. But in any case we can agree that there are a lot of papers with little to no value. And as I see it is because psychology and sociology are on the quest of creating the Method. The Method of research that works in their discipline. They do better now, then it was before. Now a paper like Zimbardo's wouldn't stand a chance. Papers from 30 years ago probably wouldn't seem rigorous enough now. The Method is becoming better, and probably it will become even better in a future and today's research will be deemed not good enough to be science, and will be read by undergraduates as a cautionary tale how one mustn't do Science. It is a little like programming which changes with each passing decade, though a little slower, I'd guess. But socially it is different. I mean if we look not at results of research but on a social processes leading to this result, it is different from programming. Programmers are expected to create programs that work, they need to deliver. There is a final clear criterion of a success. Psychologists do not have such a criterion. They seek Truth, but what Truth is and what isn't is debatable. And they work for grants, not for revenue. Programmers are inventing new tools and techniques allowing to create better or more complex programs, but at the same time they also need to deliver something useful for non-programmers. Psychologists are inventing new tools and techniques of research (or sharpen old ones), but they do not feel the urge to deliver something to non-researchers. And to be honest oftentimes they are unable to deliver. They can find some grain of truth, but no one can find a practical application for this grain yet. I think it will get better when neuroscience, AI, and psychology converge and some kind of a psychological theory will emerge. They are doing it now, and they are bound to collide at some point in a future. There are lots of psychology theories now, but no physicist will agree that any of them deserves to be called a theory. They are more like vague descriptions than theories. Or they are talk about some small part of mind. I cannot say about prospects of sociology because I know little about it, but I'd say it would take more time to bring sociology to a high standards of an empirical science. > Trying to correlate the delay between eating a marshmellow as a child and income as an adult is ridiculous, yet collectively we have spent untold money and time on tenured professors, conferences, papers, and pop psychology books about such things. Another example is Power Posing. How Amy Cuddy still gets employment is a mystery. May be it is ridiculous. But if we want to know how human mind works, we better pay for it. Researchers played with electricity for centuries. Michael Faraday himself spent half a century designing experiments. All that didn't deliver nothing of a real practical value. But then James Clerk Maxwell published his equations. And suddenly electricity started to deliver. Electricity delivered probably more than any other technology.

spritefs 4y ago

> I would take it even farther and claim that very little in experimental psychology / sociology has any value whatsoever. Oh no they definitely have value, in the same way that metaphysics has value

mattkrause 4y ago

“Experimental psychology” might be a bigger field than you imagine. It includes tons of work characterizing sensory perception. These studies tend to replicate pretty well and they have a pretty broad range of real-world applications: psychoacoustic models for audio compression, principles like pop-out and serial/parallel search that inform user interfaces, color reproduction. In fact, psychophysical experiments pinned down the properties of cone cells over a hundred years before they were directly measured. Psychometrics, the study of test/assessment design, also shows up all over the place: education, obviously, but also marketing and other forms of evaluation.

melagonster 4y ago

the example of marshmellow was modified by original researcher. reversely, this is a example of good research.