4 ms·

Lesson - just because people claim they are doing "science" doesn't mean they are doing either good or correct science. Subtle unchallenged assumptions, cause-a

by tus89 5y ago

Lesson - just because people claim they are doing "science" doesn't mean they are doing either good or correct science. Subtle unchallenged assumptions, cause-and-effect confusion, inadequate root-cause analysis, invalid inferential logic and questionable proof-by-statistics. Maybe a better question - should psychology even be considered scientific or a science in the first place?

undreren 5y ago
Sloppy science is done in every field, natural sciences included. The article even leads with the statement that psychology experience more reversals due to being exceptionally open in terms of sharing code and data compared to other social sciences. Reversals through replication failure is scientific progress. Your “better” question is nothing of the sort, just lazy contrarianism.
- RcouF1uZ4gsC 5y ago
  > Reversals through replication failure is scientific progress. Not really. If a flat earther comes and says that the earth is flat, and you take him up in a airplane and show him the curvature of the earth, he can’t say that flat earth study is science because “reversals through replication failure is scientific progress” There is a difference between making real scientific contributions through careful research, trial design, and statistical analysis, and spouting BS. After a bunch of high profile reversals, psychology is looking more like the spouting BS group that then justifies themselves by saying that clearing up BS is scientific advancement. If we want society to be able to actually use science to make decisions, then we have to be careful to differentiate science from BS.
  - derbOac 5y ago
    https://www.physicsforums.com/threads/shinichi-mochizukis-abc-conjecture-and-replication-crisis-in-maths.1002682/ https://www.physicsforums.com/threads/shinichi-mochizukis-ab...
- mcswell 5y ago
  Two fields which claim some degree of scientific approach which are probably (IMO) worse than psychology: political science and anthropology. I of course exempt my own field, linguistics, which is the most scientific of all. (Yes, that is a joke.)
BurningFrog 5y ago
Even if people have impressive titles at institutions and landmark discoveries to their name, doesn't mean they are doing either good or correct science.
BurningFrog 5y ago
> should psychology even be considered scientific or a science in the first place? You can do good scientific research in that field. But it's much harder than in other fields.
- glial 5y ago
  The inside joke is that psychology is not a hard science, but is the hardest science.
  - BurningFrog 5y ago
    Brilliant!
- exporectomy 5y ago
  I wish psychology researchers would organize bigger experiments more. Instead of 1000 sample sizes of 20, why not have one sample size of 20000? Less personal glory, and more hard management work, no doubt, but the preponderance of small studies seems like a huge waste.
  - glial 5y ago
    Most psychology work is done in universities with limited funding. Hiring the staff to run a study with 20,000 people is prohibitively expensive for any lab. Some alternatives exist - for example, some studies can be run on Amazon Mechanical Turk. Tom Griffiths using this to good effect. (& I did the same for my dissertation)
throwawaysea 5y ago
Everything you say is true, probably for all fields. There is undeserved faith in peer review processes (https://www.vox.com/2015/12/7/9865086/peer-review-science-problems https://www.vox.com/2015/12/7/9865086/peer-review-science-pr...), inadequate understanding of statistics (or maybe purposefully poor application of statistics), poor incentive structures, and other issues that corrupt "science". This is why I always cringe a little when someone says "trust the science". Yet at the same time, I think psychology is especially susceptible to these problems, because it is a more fuzzy "social science" (more like sociology, less like physics). It suffers immense bias due to those who select themselves into the field and also how they study it. Psychology researchers typically conduct non-generalizable experiments with immense sampling bias (https://www.psychologytoday.com/us/blog/non-weird-science/202004/psychologys-weird-problem https://www.psychologytoday.com/us/blog/non-weird-science/20...). Then there are replication problems (https://www.theatlantic.com/science/archive/2018/11/psychologys-replication-crisis-real/576223/ https://www.theatlantic.com/science/archive/2018/11/psycholo...). They're also not making any progress in addressing these issues because of meta cultural problems about acknowledging problems in the field (https://www.wired.com/2016/03/psychology-crisis-whether-crisis/ https://www.wired.com/2016/03/psychology-crisis-whether-cris...).
derbOac 5y ago
Killing the messenger, really. This stuff has been documented in all sorts of fields, including pharmacology, neuroscience, oncology, you name it. I think there's been posts here on HN lately about replicability problems with AI research? Psychology is fuzzy because of the subject matter, but (appropriately I think) it's also better at turning the microscope on itself (meta-analysis really has its origins in psychology). Pretending this doesn't happen elsewhere is dangerous. Maybe the rates vary from field to field, but psychology isn't alone. If you applied that standard to every field there would be almost nothing left except maybe physics and some other closely related fields.
deleted 5y ago
[deleted]
analog31 5y ago
>>>> Maybe a better question - should psychology even be considered scientific or a science in the first place? Psychology has embraced scientific methodology. However, it has struggled to produce a reliable scientific knowledge base. There are some successes in psychology, that tempt one to apply the label "scientific," but they are in areas that might be controversial, such as advertising and manipulation.
- tus89 5y ago
  > Psychology has embraced scientific methodology. I refer you to my statement about "unchallenged assumptions". Like the assumption that the reductive scientific process can be glibly applied to emergent complex phenomenon like psychology. As I said, people can say they are doing science, it neither means what they are doing is good or correct science.
  - coolgeek 5y ago
    Not sure where you're getting your definition of science from, but it doesn't correspond to those from authoritative sources. Here, for example, is Brittanica's: > Science, any system of knowledge that is concerned with the physical world and its phenomena and that entails unbiased observations and systematic experimentation. In general, a science involves a pursuit of knowledge covering general truths or the operations of fundamental laws. > Science can be divided into different branches based on the subject of study. The physical sciences study the inorganic world and comprise the fields of astronomy, physics, chemistry, and the Earth sciences. The biological sciences such as biology and medicine study the organic world of life and its processes. Social sciences like anthropology and economics study the social and cultural aspects of human behaviour. https://www.britannica.com/science/science https://www.britannica.com/science/science This article has been on the web since 1998. The Encyclopedia from which it is sourced has been publishing in print since 1768. If the entry on Science wasn't in the first edition, it probably wasn't too long before it first appeared. I encourage you to take your issue up with the editors and authors there. https://corporate.britannica.com/contact/ https://corporate.britannica.com/contact/ Do let us know how you make out!

by tus89 5y ago

undreren 5y ago

Sloppy science is done in every field, natural sciences included. The article even leads with the statement that psychology experience more reversals due to being exceptionally open in terms of sharing code and data compared to other social sciences. Reversals through replication failure is scientific progress. Your “better” question is nothing of the sort, just lazy contrarianism.

RcouF1uZ4gsC 5y ago
> Reversals through replication failure is scientific progress. Not really. If a flat earther comes and says that the earth is flat, and you take him up in a airplane and show him the curvature of the earth, he can’t say that flat earth study is science because “reversals through replication failure is scientific progress” There is a difference between making real scientific contributions through careful research, trial design, and statistical analysis, and spouting BS. After a bunch of high profile reversals, psychology is looking more like the spouting BS group that then justifies themselves by saying that clearing up BS is scientific advancement. If we want society to be able to actually use science to make decisions, then we have to be careful to differentiate science from BS.
- derbOac 5y ago
  https://www.physicsforums.com/threads/shinichi-mochizukis-abc-conjecture-and-replication-crisis-in-maths.1002682/ https://www.physicsforums.com/threads/shinichi-mochizukis-ab...
mcswell 5y ago
Two fields which claim some degree of scientific approach which are probably (IMO) worse than psychology: political science and anthropology. I of course exempt my own field, linguistics, which is the most scientific of all. (Yes, that is a joke.)

BurningFrog 5y ago

Even if people have impressive titles at institutions and landmark discoveries to their name, doesn't mean they are doing either good or correct science.

BurningFrog 5y ago

> should psychology even be considered scientific or a science in the first place? You can do good scientific research in that field. But it's much harder than in other fields.

glial 5y ago
The inside joke is that psychology is not a hard science, but is the hardest science.
- BurningFrog 5y ago
  Brilliant!
exporectomy 5y ago
I wish psychology researchers would organize bigger experiments more. Instead of 1000 sample sizes of 20, why not have one sample size of 20000? Less personal glory, and more hard management work, no doubt, but the preponderance of small studies seems like a huge waste.
- glial 5y ago
  Most psychology work is done in universities with limited funding. Hiring the staff to run a study with 20,000 people is prohibitively expensive for any lab. Some alternatives exist - for example, some studies can be run on Amazon Mechanical Turk. Tom Griffiths using this to good effect. (& I did the same for my dissertation)

throwawaysea 5y ago

Everything you say is true, probably for all fields. There is undeserved faith in peer review processes (https://www.vox.com/2015/12/7/9865086/peer-review-science-problems https://www.vox.com/2015/12/7/9865086/peer-review-science-pr...), inadequate understanding of statistics (or maybe purposefully poor application of statistics), poor incentive structures, and other issues that corrupt "science". This is why I always cringe a little when someone says "trust the science". Yet at the same time, I think psychology is especially susceptible to these problems, because it is a more fuzzy "social science" (more like sociology, less like physics). It suffers immense bias due to those who select themselves into the field and also how they study it. Psychology researchers typically conduct non-generalizable experiments with immense sampling bias (https://www.psychologytoday.com/us/blog/non-weird-science/202004/psychologys-weird-problem https://www.psychologytoday.com/us/blog/non-weird-science/20...). Then there are replication problems (https://www.theatlantic.com/science/archive/2018/11/psychologys-replication-crisis-real/576223/ https://www.theatlantic.com/science/archive/2018/11/psycholo...). They're also not making any progress in addressing these issues because of meta cultural problems about acknowledging problems in the field (https://www.wired.com/2016/03/psychology-crisis-whether-crisis/ https://www.wired.com/2016/03/psychology-crisis-whether-cris...).

derbOac 5y ago

Killing the messenger, really. This stuff has been documented in all sorts of fields, including pharmacology, neuroscience, oncology, you name it. I think there's been posts here on HN lately about replicability problems with AI research? Psychology is fuzzy because of the subject matter, but (appropriately I think) it's also better at turning the microscope on itself (meta-analysis really has its origins in psychology). Pretending this doesn't happen elsewhere is dangerous. Maybe the rates vary from field to field, but psychology isn't alone. If you applied that standard to every field there would be almost nothing left except maybe physics and some other closely related fields.

deleted 5y ago

[deleted]

analog31 5y ago

>>>> Maybe a better question - should psychology even be considered scientific or a science in the first place? Psychology has embraced scientific methodology. However, it has struggled to produce a reliable scientific knowledge base. There are some successes in psychology, that tempt one to apply the label "scientific," but they are in areas that might be controversial, such as advertising and manipulation.

tus89 5y ago
> Psychology has embraced scientific methodology. I refer you to my statement about "unchallenged assumptions". Like the assumption that the reductive scientific process can be glibly applied to emergent complex phenomenon like psychology. As I said, people can say they are doing science, it neither means what they are doing is good or correct science.
- coolgeek 5y ago
  Not sure where you're getting your definition of science from, but it doesn't correspond to those from authoritative sources. Here, for example, is Brittanica's: > Science, any system of knowledge that is concerned with the physical world and its phenomena and that entails unbiased observations and systematic experimentation. In general, a science involves a pursuit of knowledge covering general truths or the operations of fundamental laws. > Science can be divided into different branches based on the subject of study. The physical sciences study the inorganic world and comprise the fields of astronomy, physics, chemistry, and the Earth sciences. The biological sciences such as biology and medicine study the organic world of life and its processes. Social sciences like anthropology and economics study the social and cultural aspects of human behaviour. https://www.britannica.com/science/science https://www.britannica.com/science/science This article has been on the web since 1998. The Encyclopedia from which it is sourced has been publishing in print since 1768. If the entry on Science wasn't in the first edition, it probably wasn't too long before it first appeared. I encourage you to take your issue up with the editors and authors there. https://corporate.britannica.com/contact/ https://corporate.britannica.com/contact/ Do let us know how you make out!