5 ms·

And that is why I don't consider psychology as a whole a science. It simply isn't. Imagine if something like this happened in physics, or mathematics. It would

And that is why I don't consider psychology as a whole a science. It simply isn't. Imagine if something like this happened in physics, or mathematics. It would have huge impacts on all work that cited it. In psychology, not so much. Does this mean psychology is useless? I don't think so, there have been many people helped by application of psychology, but the sooner we take it out of the same category as for example mathematics or medicine the sooner we can move on. I'd love to be convinced otherwise that we as humanity have a chance of understanding ourselves to the point where we can draw meaningful conclusions about human happiness or other feelings/states of mind. However, I think at this stage our chance of accomplishing this is similar to a medieval blacksmith doing a successful root canal treatment on of his dental patients. I'm definitely looking at the advancements in AI with hope that perhaps replicating our own thought processes in machines will enable us to study them more systematically and to understand them, but these are still early days.

bakuninsbart 3y ago
It already is in a completely different category than mathematics; mathematics is a humanity while psychology is a science. Do the standards need to be improved in psychology? Almost certainly, and this rings true for most quantitative social sciences. But research should be evaluated on its own, not by field. There is still a lot of good, solid knowledge being produced through the scientific methods in these fields. We just generally lack the unifying theories we know from eg. physics, because the subject matter is so much more complicated, and the field is much younger. And in a lot of ways, the way academia works is not conductive to producing high-quality, impactful research in these fields.
- whimsicalism 3y ago
  Mathematics is not a humanities, it is a formal science, unless you are using some very idiosyncratic definition.
  - beojan 3y ago
    Mathematics isn't a humanity but it isn't a science either. It's purer than a science, demanding logical proof, not just evidence.
    - xdavidliu 3y ago
      reminds me of a quote from Sipser's Theory of Computation textbook, towards the beginning. He says something like "in mathematics, not even evidence is enough."
  - bee_rider 3y ago
    Most mathematics is built on pure reasoning, not experiments. Calling it a science is a bit of a downgrade.
  - bakuninsbart 3y ago
    Fair, I'd say it is better to be idosyncratic and consistent than conforming and incongruent. "Formal science" does not make any sense as a subcategory of science if the definition of science basically omits anything that makes mathematics and philosophy what it is. The "formal sciences" do not use the scientific method, and are basically completely distinct from the natural and social sciences in the ways they generate knowledge. In my language, it is generally linked to the humanities, but I'm not hard set on that.
jdietrich 3y ago
Psychology is really the tip of the iceberg when it comes to research misconduct. We've noticed a lot of dodgy papers because of the broad public interest in the field and the overlap with the much more rigorous field of psychiatry, but I have no doubt that there are scores of Stapels and Arielys in most fields of research. In physics, we've seen an immense amount of recent controversy over LK-99; while the material might have some interesting properties, it is almost certainly not the room-temperature ambient-pressure superconductor that Lee and Kim claim it to be. How many totally bogus physics papers might be lurking in the literature, ignored and mostly un-cited? We don't know, because most of those papers don't make sufficiently interesting or exciting claims to inspire a raft of replication attempts. For all our aspirational beliefs about the scientific method and the power of peer review, most papers are subject to little more than a superficial sniff test. There are plausible reasons to believe that psychology might have a worse misconduct problem than other fields, but the idea that psychology is uniquely broken is, I fear, simply wishful thinking.
- DonsDiscountGas 3y ago
  That was some dodgy science (and people not being skeptical enough) but at least nobody out-right lied (afaik). And the hype only lasted a few weeks. The Schon case is worse IMHO and similarly points to bad incentives, although at least that one is 20 years old and there hasn't been a similarly large-scale fraud since (again afaik). https://en.wikipedia.org/wiki/Sch%C3%B6n_scandal https://en.wikipedia.org/wiki/Sch%C3%B6n_scandal
cycomanic 3y ago
It's popular here (and in other more "technically" inclined communities) to dunk on the "soft sciences", but the hard sciences also had their share of fraud cases (the Schön revelations being some of the most prominent). The discovery of Schön's fraud also didn't lead to theories suddenly being rewritten. This is simply a misunderstanding of how science works and the contribution/impact of a single scientist. In the end sure some minor directions will be impacted, people will see conclusions still hold, but overall theories will only marginally be impacted. Something only becomes well accepted theory if it can be reproduced again and again, fraud will come out eventually. That is the beauty of the scientific process.
- lofatdairy 3y ago
  Definitely agree that this is not just a psychology issue[^1]. I'm not quite as optimistic about the implications, however. The way academia operates is very trust heavy. This makes sense, no lab can be experts in all fields of study, and perform every experiment themselves. If something doesn't work for you, the default assumption is that something in your system is incompatible with the previous work, or you're just doing something wrong. So sometimes it takes quite a concerted effort to identify bad science, because legitimate work can be nudged towards supporting + compatible with this bad science. You basically have no choice but to have basis in what is widely believed to be true if you want grant funding or papers published. From a purely theoretical/knowledge standpoint, I think I agree - the truth will eventually be accepted. But in fields like biology, this can be at the cost of drugs not working, actually promising avenues not explored, wasted resources - tangible harms. [^1]: E.g. the recent Alzheimer's controversy, Stanford's president scandal, 5-HTTLPR, etc. Ironically, Ronald Fisher argues that Gregor Mendel's work, the foundation of modern biology, to be essentially fraudulent despite being correct.
  - thaumasiotes 3y ago
    > Ironically, Ronald Fisher argues that Gregor Mendel's work, the foundation of modern biology, to be essentially fraudulent despite being correct. There's nothing weird about that. You'll see exactly the same phenomenon by going through a low-level mathematics textbooks and noticing how many of the problems have small integer solutions. Does that happen in real problems? No. Does the fact that all the problems are fraudulent mean that the math they're meant to teach you is wrong? No. The math is right; that's why the problems were written up the way they were.
gr__or 3y ago
https://smallpotatoes.paulbloom.net/p/psychology-is-ok https://smallpotatoes.paulbloom.net/p/psychology-is-ok

bakuninsbart 3y ago

It already is in a completely different category than mathematics; mathematics is a humanity while psychology is a science. Do the standards need to be improved in psychology? Almost certainly, and this rings true for most quantitative social sciences. But research should be evaluated on its own, not by field. There is still a lot of good, solid knowledge being produced through the scientific methods in these fields. We just generally lack the unifying theories we know from eg. physics, because the subject matter is so much more complicated, and the field is much younger. And in a lot of ways, the way academia works is not conductive to producing high-quality, impactful research in these fields.

whimsicalism 3y ago
Mathematics is not a humanities, it is a formal science, unless you are using some very idiosyncratic definition.
- beojan 3y ago
  Mathematics isn't a humanity but it isn't a science either. It's purer than a science, demanding logical proof, not just evidence.
  - xdavidliu 3y ago
    reminds me of a quote from Sipser's Theory of Computation textbook, towards the beginning. He says something like "in mathematics, not even evidence is enough."
- bee_rider 3y ago
  Most mathematics is built on pure reasoning, not experiments. Calling it a science is a bit of a downgrade.
- bakuninsbart 3y ago
  Fair, I'd say it is better to be idosyncratic and consistent than conforming and incongruent. "Formal science" does not make any sense as a subcategory of science if the definition of science basically omits anything that makes mathematics and philosophy what it is. The "formal sciences" do not use the scientific method, and are basically completely distinct from the natural and social sciences in the ways they generate knowledge. In my language, it is generally linked to the humanities, but I'm not hard set on that.

jdietrich 3y ago

Psychology is really the tip of the iceberg when it comes to research misconduct. We've noticed a lot of dodgy papers because of the broad public interest in the field and the overlap with the much more rigorous field of psychiatry, but I have no doubt that there are scores of Stapels and Arielys in most fields of research. In physics, we've seen an immense amount of recent controversy over LK-99; while the material might have some interesting properties, it is almost certainly not the room-temperature ambient-pressure superconductor that Lee and Kim claim it to be. How many totally bogus physics papers might be lurking in the literature, ignored and mostly un-cited? We don't know, because most of those papers don't make sufficiently interesting or exciting claims to inspire a raft of replication attempts. For all our aspirational beliefs about the scientific method and the power of peer review, most papers are subject to little more than a superficial sniff test. There are plausible reasons to believe that psychology might have a worse misconduct problem than other fields, but the idea that psychology is uniquely broken is, I fear, simply wishful thinking.

DonsDiscountGas 3y ago
That was some dodgy science (and people not being skeptical enough) but at least nobody out-right lied (afaik). And the hype only lasted a few weeks. The Schon case is worse IMHO and similarly points to bad incentives, although at least that one is 20 years old and there hasn't been a similarly large-scale fraud since (again afaik). https://en.wikipedia.org/wiki/Sch%C3%B6n_scandal https://en.wikipedia.org/wiki/Sch%C3%B6n_scandal

cycomanic 3y ago

It's popular here (and in other more "technically" inclined communities) to dunk on the "soft sciences", but the hard sciences also had their share of fraud cases (the Schön revelations being some of the most prominent). The discovery of Schön's fraud also didn't lead to theories suddenly being rewritten. This is simply a misunderstanding of how science works and the contribution/impact of a single scientist. In the end sure some minor directions will be impacted, people will see conclusions still hold, but overall theories will only marginally be impacted. Something only becomes well accepted theory if it can be reproduced again and again, fraud will come out eventually. That is the beauty of the scientific process.

lofatdairy 3y ago
Definitely agree that this is not just a psychology issue[^1]. I'm not quite as optimistic about the implications, however. The way academia operates is very trust heavy. This makes sense, no lab can be experts in all fields of study, and perform every experiment themselves. If something doesn't work for you, the default assumption is that something in your system is incompatible with the previous work, or you're just doing something wrong. So sometimes it takes quite a concerted effort to identify bad science, because legitimate work can be nudged towards supporting + compatible with this bad science. You basically have no choice but to have basis in what is widely believed to be true if you want grant funding or papers published. From a purely theoretical/knowledge standpoint, I think I agree - the truth will eventually be accepted. But in fields like biology, this can be at the cost of drugs not working, actually promising avenues not explored, wasted resources - tangible harms. [^1]: E.g. the recent Alzheimer's controversy, Stanford's president scandal, 5-HTTLPR, etc. Ironically, Ronald Fisher argues that Gregor Mendel's work, the foundation of modern biology, to be essentially fraudulent despite being correct.
- thaumasiotes 3y ago
  > Ironically, Ronald Fisher argues that Gregor Mendel's work, the foundation of modern biology, to be essentially fraudulent despite being correct. There's nothing weird about that. You'll see exactly the same phenomenon by going through a low-level mathematics textbooks and noticing how many of the problems have small integer solutions. Does that happen in real problems? No. Does the fact that all the problems are fraudulent mean that the math they're meant to teach you is wrong? No. The math is right; that's why the problems were written up the way they were.

gr__or 3y ago

https://smallpotatoes.paulbloom.net/p/psychology-is-ok https://smallpotatoes.paulbloom.net/p/psychology-is-ok