4 ms·

PSA: There's also a severe replication crisis in the hard sciences. The high horse is not well warranted.

chairhairair 2y ago
Completely false equivalence. The entire foundations of modern Psychology are wobbling. In order for the same to be true for the hard sciences we would need to be failing to replicate experiments which hinge on germ theory, atomic theory, the standard model, etc. Nothing like that is happening. This false equivalence originates from several types of people: 1. Journalists that want/need to foment the largest possible catastrophe. 2. Political pundits which want/need to discredit some field. 3. Social scientists playing defense.
- llamaimperative 2y ago
  What exactly do you think are the foundations of modern psychology? Serious question. There are tons of non-replicable findings way, way further down the stack than psychology, and those tend to have a lot more relying on them than psychology/sociology studies. If you're upset about scientific validity, consider directing your ire to where problems are more likely to actually hurt people -- the "hard sciences." Nice ad hominem but I'm none of those things. I work in clinical trials, one of the few areas where we actually do have to know things, and a very good empirical demonstration of exactly how incredibly difficult that is.
  - elliotto 2y ago
    I'm curious to hear your perspective on the validity of psychology / psychiatry / sociology as someone adjacent to the field. I am a hard science maths / data science guy, but unlike a lot of my peers I have a great interest in softer reasoning (philosophy, ethics, political science etc). But I am constantly disappointed by how tainted by ideology psychology and psychiatry feel (and economics, but this is a different discussion). Do you think that psychology and psychiatry are held to the same rigour as harder sciences and should be considered as valid?
    - llamaimperative 2y ago
      All fields of inquiry are tainted by ideology. Read the history of literally any scientific field ever. The entire system is designed to accept this as fact, because science is done by humans, and to still arrive at the truth nonetheless. If we take those two observations: a) science is done by humans and b) humans have motivations, obviously the way to arrive at truth is to allow for most things to be wrong most of the time. This is the process by which we've learned every single thing about the universe. I don't know what you mean by "held to the same rigor." I don't think any psychologist on the planet would tell you we understand psychology as well as we understand basic chemical reactions.
- darby_nine 2y ago
  > The entire foundations of modern Psychology are wobbling. What do you mean by this? the field of psychology is perfectly capable of policing itself, and it's rejected much of the conclusions of its historical predecessors. > In order for the same to be true for the hard sciences we would need to be failing to replicate experiments which hinge on germ theory, atomic theory, the standard model, etc. "hard sciences" also fail to produce results relevant to most people. Sure, they can maybe make better batteries, but how can they explain how dysfunctional society is? EDIT: We can also directly blame the poor communicational skills of "hard sciences" for diet culture. "hard sciences" have failed in their efforts to produce a population that can reason about nutrition in an evidence-backed manner, and this can be traced directly to how scientists choose to present their data.
  - zarathustreal 2y ago
    You don’t need science to explain how dysfunctional society is. The problem with “soft sciences” is that they can’t produce provably correct information. Incorrect information is worse than no information.
    - darby_nine 2y ago
      > You don’t need science to explain how dysfunctional society is. You absolutely do lol, or you're just straight wrong. Take your pick. > The problem with “soft sciences” is that they can’t produce provably correct information. Neither can hard sciences. Proofs are incompatible with empiricism. Abductive proofs (which is what the scientific process offers) are necessarily bounded by limited certainty. All you can do is progressively improve certainty approaching 100%, but reaching it is necessarily impossible. This is just basic Hume. You can never be 100% sure the sun will rise tomorrow or that the fundamental laws of physics won't arbitrarily change. But, the same applies to soft sciences. We can and do increase our certainty continually. This is absolutely worthwhile and is probably far more valuable to humanity than merely modelling physical phenomena.
      - zarathustreal 2y ago
- kjkjadksj 2y ago
  To be fair all of that was up in question a century ago in the hard sciences. People used to believe in the plum pudding model, or doubt what component in the cell contained genetic information. The only thing that changed that was incremental experimental evidence. Social sciences are evidently going through a similar transition but we shouldn’t use that to discredit the field alone, work is still able to stand for itself if it is done well.
  - llamaimperative 2y ago
    The level of ignorance of this basic, basic historical fact is just completely astounding on this website sometimes. It was called natural philosophy for Christ's sake. Modern medicine's forefather was heroic medicine, based on modulating the 4 humors (blood, phlegm, yellow bile, black bile). We didn't zap into existence with the hard sciences being hard. We made them that way through centuries of intellectual effort, almost all of which turned out to be wrong!
enavari 2y ago
True, but I remember reading the replication failure rates were twice as much in the social science than in medicine.
- matthewdgreen 2y ago
  We constantly see small medical studies (<100 participants) posted here on HN that produce exciting results, which then disappear from view and/or fall apart when replicated with larger cohorts.
  - llamaimperative 2y ago
    How is this different from science working?
    - kjkjadksj 2y ago
      When it happens in the hard sciences its science working, when it happens in the soft sciences its a replication crisis.
      - llamaimperative 2y ago
        Haha that seems to be the case!
  - pessimizer 2y ago
    If that didn't happen, it would mean that math was broken. That's how statistics is supposed to work.
noslenwerdna 2y ago
Depends what you count as hard science. The replication rate in high energy particle physics is near 100%? When the LHC started up they were able to measure nearly all of the particle resonances found in the 20th century. It's not like they suddenly disproved the existence of electrons or something.
- llamaimperative 2y ago
  Sure, further down the stack of chaos that is the universe (physics -> chemistry -> biology -> psychology -> sociology), it's much easier to conduct controlled experiments. That doesn't mean the people engaged in research at the bottom of the stack are good and the people at the top of the stack are bad. Nor does it mean we shouldn't be trying our best to understand things near the top of the stack.
  - superposeur 2y ago
    > it's much easier to conduct controlled experiments. Very true. But this means more statistics and controls are necessary to get solid result from a social science experiment then a particle physics experiment, no? Clearly, this is practically impossible, but there you go.
    - llamaimperative 2y ago
      > Clearly, this is practically impossible No it's not? You put more money into the studies and you can do bigger, better versions of them. A major obstacle to putting more money into studies: people jerking themselves off about how soft sciences are a joke and hard sciences are Super Serious Business.
      - noslenwerdna 2y ago
        But why do those fields deserve more money, when at least a large part of the problem is cultural. One example is the famous reluctance to publish negative results in psychology. Nearly all published results in (collider) particle physics are negative. If senior faculty prefer to only hire people with a string of published postive findings, you are literally encouraging p-hacking. Again, they are not "bad" people, it is just that the system the senior people have setup in that field is not conducive to doing good science.

Completely false equivalence. The entire foundations of modern Psychology are wobbling. In order for the same to be true for the hard sciences we would need to be failing to replicate experiments which hinge on germ theory, atomic theory, the standard model, etc. Nothing like that is happening. This false equivalence originates from several types of people: 1. Journalists that want/need to foment the largest possible catastrophe. 2. Political pundits which want/need to discredit some field. 3. Social scientists playing defense.

llamaimperative 2y ago
What exactly do you think are the foundations of modern psychology? Serious question. There are tons of non-replicable findings way, way further down the stack than psychology, and those tend to have a lot more relying on them than psychology/sociology studies. If you're upset about scientific validity, consider directing your ire to where problems are more likely to actually hurt people -- the "hard sciences." Nice ad hominem but I'm none of those things. I work in clinical trials, one of the few areas where we actually do have to know things, and a very good empirical demonstration of exactly how incredibly difficult that is.
- elliotto 2y ago
  I'm curious to hear your perspective on the validity of psychology / psychiatry / sociology as someone adjacent to the field. I am a hard science maths / data science guy, but unlike a lot of my peers I have a great interest in softer reasoning (philosophy, ethics, political science etc). But I am constantly disappointed by how tainted by ideology psychology and psychiatry feel (and economics, but this is a different discussion). Do you think that psychology and psychiatry are held to the same rigour as harder sciences and should be considered as valid?
  - llamaimperative 2y ago
    All fields of inquiry are tainted by ideology. Read the history of literally any scientific field ever. The entire system is designed to accept this as fact, because science is done by humans, and to still arrive at the truth nonetheless. If we take those two observations: a) science is done by humans and b) humans have motivations, obviously the way to arrive at truth is to allow for most things to be wrong most of the time. This is the process by which we've learned every single thing about the universe. I don't know what you mean by "held to the same rigor." I don't think any psychologist on the planet would tell you we understand psychology as well as we understand basic chemical reactions.
darby_nine 2y ago
> The entire foundations of modern Psychology are wobbling. What do you mean by this? the field of psychology is perfectly capable of policing itself, and it's rejected much of the conclusions of its historical predecessors. > In order for the same to be true for the hard sciences we would need to be failing to replicate experiments which hinge on germ theory, atomic theory, the standard model, etc. "hard sciences" also fail to produce results relevant to most people. Sure, they can maybe make better batteries, but how can they explain how dysfunctional society is? EDIT: We can also directly blame the poor communicational skills of "hard sciences" for diet culture. "hard sciences" have failed in their efforts to produce a population that can reason about nutrition in an evidence-backed manner, and this can be traced directly to how scientists choose to present their data.
- zarathustreal 2y ago
  You don’t need science to explain how dysfunctional society is. The problem with “soft sciences” is that they can’t produce provably correct information. Incorrect information is worse than no information.
  - darby_nine 2y ago
    > You don’t need science to explain how dysfunctional society is. You absolutely do lol, or you're just straight wrong. Take your pick. > The problem with “soft sciences” is that they can’t produce provably correct information. Neither can hard sciences. Proofs are incompatible with empiricism. Abductive proofs (which is what the scientific process offers) are necessarily bounded by limited certainty. All you can do is progressively improve certainty approaching 100%, but reaching it is necessarily impossible. This is just basic Hume. You can never be 100% sure the sun will rise tomorrow or that the fundamental laws of physics won't arbitrarily change. But, the same applies to soft sciences. We can and do increase our certainty continually. This is absolutely worthwhile and is probably far more valuable to humanity than merely modelling physical phenomena.
    - zarathustreal 2y ago
kjkjadksj 2y ago
To be fair all of that was up in question a century ago in the hard sciences. People used to believe in the plum pudding model, or doubt what component in the cell contained genetic information. The only thing that changed that was incremental experimental evidence. Social sciences are evidently going through a similar transition but we shouldn’t use that to discredit the field alone, work is still able to stand for itself if it is done well.
- llamaimperative 2y ago
  The level of ignorance of this basic, basic historical fact is just completely astounding on this website sometimes. It was called natural philosophy for Christ's sake. Modern medicine's forefather was heroic medicine, based on modulating the 4 humors (blood, phlegm, yellow bile, black bile). We didn't zap into existence with the hard sciences being hard. We made them that way through centuries of intellectual effort, almost all of which turned out to be wrong!

enavari 2y ago

True, but I remember reading the replication failure rates were twice as much in the social science than in medicine.

matthewdgreen 2y ago
We constantly see small medical studies (<100 participants) posted here on HN that produce exciting results, which then disappear from view and/or fall apart when replicated with larger cohorts.
- llamaimperative 2y ago
  How is this different from science working?
  - kjkjadksj 2y ago
    When it happens in the hard sciences its science working, when it happens in the soft sciences its a replication crisis.
    - llamaimperative 2y ago
      Haha that seems to be the case!
- pessimizer 2y ago
  If that didn't happen, it would mean that math was broken. That's how statistics is supposed to work.

noslenwerdna 2y ago

Depends what you count as hard science. The replication rate in high energy particle physics is near 100%? When the LHC started up they were able to measure nearly all of the particle resonances found in the 20th century. It's not like they suddenly disproved the existence of electrons or something.

llamaimperative 2y ago
Sure, further down the stack of chaos that is the universe (physics -> chemistry -> biology -> psychology -> sociology), it's much easier to conduct controlled experiments. That doesn't mean the people engaged in research at the bottom of the stack are good and the people at the top of the stack are bad. Nor does it mean we shouldn't be trying our best to understand things near the top of the stack.
- superposeur 2y ago
  > it's much easier to conduct controlled experiments. Very true. But this means more statistics and controls are necessary to get solid result from a social science experiment then a particle physics experiment, no? Clearly, this is practically impossible, but there you go.
  - llamaimperative 2y ago
    > Clearly, this is practically impossible No it's not? You put more money into the studies and you can do bigger, better versions of them. A major obstacle to putting more money into studies: people jerking themselves off about how soft sciences are a joke and hard sciences are Super Serious Business.
    - noslenwerdna 2y ago
      But why do those fields deserve more money, when at least a large part of the problem is cultural. One example is the famous reluctance to publish negative results in psychology. Nearly all published results in (collider) particle physics are negative. If senior faculty prefer to only hire people with a string of published postive findings, you are literally encouraging p-hacking. Again, they are not "bad" people, it is just that the system the senior people have setup in that field is not conducive to doing good science.