3 ms·

Depends what you count as hard science. The replication rate in high energy particle physics is near 100%? When the LHC started up they were able to measure nea

Depends what you count as hard science. The replication rate in high energy particle physics is near 100%? When the LHC started up they were able to measure nearly all of the particle resonances found in the 20th century. It's not like they suddenly disproved the existence of electrons or something.

llamaimperative 2y ago
Sure, further down the stack of chaos that is the universe (physics -> chemistry -> biology -> psychology -> sociology), it's much easier to conduct controlled experiments. That doesn't mean the people engaged in research at the bottom of the stack are good and the people at the top of the stack are bad. Nor does it mean we shouldn't be trying our best to understand things near the top of the stack.
- superposeur 2y ago
  > it's much easier to conduct controlled experiments. Very true. But this means more statistics and controls are necessary to get solid result from a social science experiment then a particle physics experiment, no? Clearly, this is practically impossible, but there you go.
  - llamaimperative 2y ago
    > Clearly, this is practically impossible No it's not? You put more money into the studies and you can do bigger, better versions of them. A major obstacle to putting more money into studies: people jerking themselves off about how soft sciences are a joke and hard sciences are Super Serious Business.
    - noslenwerdna 2y ago
      But why do those fields deserve more money, when at least a large part of the problem is cultural. One example is the famous reluctance to publish negative results in psychology. Nearly all published results in (collider) particle physics are negative. If senior faculty prefer to only hire people with a string of published postive findings, you are literally encouraging p-hacking. Again, they are not "bad" people, it is just that the system the senior people have setup in that field is not conducive to doing good science.
      - llamaimperative 2y ago
        > But why do those fields deserve more money Because it'd be good to understand what makes people happy, for example. Or what enables relationships to thrive. Or when different forms of government are suitable or unsuitable to solve a set of problems, etc. Sorry to break it to the hard-sciencers, but the vast majority of opportunities left in the western world to improve people's lives is not particle accelerators, it's answering questions like: "what actually helps people feel satisfied in life, loved in their relationships, and belonging in their community?" > At least a large part of the problem is cultural Is it? Why so? Negative results aren't published in almost any field, and that's actually a good on ramp to the discussion we should be having, which is about the broken incentives of science and scientific publishing specifically. The broken incentive model isn't special to softer sciences and it has far more dire consequences in other domains. You can't possibly think that soft sciences are the only ones hiring people with a string of positive results... right?
        noslenwerdna 2y ago
        I agree that studying psychology better could be beneficial. Is it possible? Or more to the point, is it merely a matter of money, as you said? I said a large part of the problem is cultural, I did not say that psychology is the only field with cultural problems. I'm not sure how you got that idea.
        llamaimperative 2y ago
        No no, what you said is that it's "clearly practically impossible" to have more statistics, more controls, etc. to get higher powered studies of high-chaos questions like the ones asked in the soft sciences. I said, to that point: no it's not. You just do bigger, longer term, more complete studies. The limiting factor on this -- right now -- is typically money. Perhaps you can pour infinite money and problems with e.g. recruitment or monitoring will still prevent us from getting to statistical power, but maybe not. That is not the only problem social sciences face, but most of the problems they face are not exclusive to social sciences whatsoever, which then prompts the obvious question of why they get so much flak.
    - smegger001 2y ago
      As my sister who is studying one of the soft sciences put it to me when i pointed out the lack of rigor compared to the hard. "sure we could make psychology a hard science but pesky ethics boards wont approve me raising batches of several hundred human clones in controlled environment for each test"
    - superposeur 2y ago
      I mean, a psych experiment will never have an N comparable to a particle physics experiment or be able to reach the 5-sigma threshold for discovery that now prevails in physics. On the other hand, the object of study for psych is intrinsically interesting since we are people and if something reliable can be gleaned then it's certainly worth money. My concern is that "bigger, better" (as you say) would have to include millions of people across cultures and times, tracking longitudinally, with randomization and controls. (Again, more complexity requires more statistics, not less.). Is this practical? Maybe ...
- noslenwerdna 2y ago
  I didn't claim that it means they are "bad" whatever that means. I am saying that we should take those claims less seriously, especially if the results from that domain don't replicate, as in the case of psychology and other social sciences. Maybe there is little we can conclusively say about those domains.
  - llamaimperative 2y ago
    Yes we're in agreement on this. There was little we could conclusively say about any domain until long term, concerted effort was made to understand each of them.
zanellato19 2y ago
Anecdotally, I have a friend that's doing bio-chem doctorate and she has said that replication rate on that is abysmal for biology, chemistry and consequently bio-chem. I'm sure some areas of physics have near 100% and some simply don't.

llamaimperative 2y ago

Sure, further down the stack of chaos that is the universe (physics -> chemistry -> biology -> psychology -> sociology), it's much easier to conduct controlled experiments. That doesn't mean the people engaged in research at the bottom of the stack are good and the people at the top of the stack are bad. Nor does it mean we shouldn't be trying our best to understand things near the top of the stack.

superposeur 2y ago
> it's much easier to conduct controlled experiments. Very true. But this means more statistics and controls are necessary to get solid result from a social science experiment then a particle physics experiment, no? Clearly, this is practically impossible, but there you go.
- llamaimperative 2y ago
  > Clearly, this is practically impossible No it's not? You put more money into the studies and you can do bigger, better versions of them. A major obstacle to putting more money into studies: people jerking themselves off about how soft sciences are a joke and hard sciences are Super Serious Business.
  - noslenwerdna 2y ago
    But why do those fields deserve more money, when at least a large part of the problem is cultural. One example is the famous reluctance to publish negative results in psychology. Nearly all published results in (collider) particle physics are negative. If senior faculty prefer to only hire people with a string of published postive findings, you are literally encouraging p-hacking. Again, they are not "bad" people, it is just that the system the senior people have setup in that field is not conducive to doing good science.
    - llamaimperative 2y ago
      > But why do those fields deserve more money Because it'd be good to understand what makes people happy, for example. Or what enables relationships to thrive. Or when different forms of government are suitable or unsuitable to solve a set of problems, etc. Sorry to break it to the hard-sciencers, but the vast majority of opportunities left in the western world to improve people's lives is not particle accelerators, it's answering questions like: "what actually helps people feel satisfied in life, loved in their relationships, and belonging in their community?" > At least a large part of the problem is cultural Is it? Why so? Negative results aren't published in almost any field, and that's actually a good on ramp to the discussion we should be having, which is about the broken incentives of science and scientific publishing specifically. The broken incentive model isn't special to softer sciences and it has far more dire consequences in other domains. You can't possibly think that soft sciences are the only ones hiring people with a string of positive results... right?
      - noslenwerdna 2y ago
        I agree that studying psychology better could be beneficial. Is it possible? Or more to the point, is it merely a matter of money, as you said? I said a large part of the problem is cultural, I did not say that psychology is the only field with cultural problems. I'm not sure how you got that idea.
        llamaimperative 2y ago
        No no, what you said is that it's "clearly practically impossible" to have more statistics, more controls, etc. to get higher powered studies of high-chaos questions like the ones asked in the soft sciences. I said, to that point: no it's not. You just do bigger, longer term, more complete studies. The limiting factor on this -- right now -- is typically money. Perhaps you can pour infinite money and problems with e.g. recruitment or monitoring will still prevent us from getting to statistical power, but maybe not. That is not the only problem social sciences face, but most of the problems they face are not exclusive to social sciences whatsoever, which then prompts the obvious question of why they get so much flak.
  - smegger001 2y ago
    As my sister who is studying one of the soft sciences put it to me when i pointed out the lack of rigor compared to the hard. "sure we could make psychology a hard science but pesky ethics boards wont approve me raising batches of several hundred human clones in controlled environment for each test"
  - superposeur 2y ago
    I mean, a psych experiment will never have an N comparable to a particle physics experiment or be able to reach the 5-sigma threshold for discovery that now prevails in physics. On the other hand, the object of study for psych is intrinsically interesting since we are people and if something reliable can be gleaned then it's certainly worth money. My concern is that "bigger, better" (as you say) would have to include millions of people across cultures and times, tracking longitudinally, with randomization and controls. (Again, more complexity requires more statistics, not less.). Is this practical? Maybe ...
noslenwerdna 2y ago
I didn't claim that it means they are "bad" whatever that means. I am saying that we should take those claims less seriously, especially if the results from that domain don't replicate, as in the case of psychology and other social sciences. Maybe there is little we can conclusively say about those domains.
- llamaimperative 2y ago
  Yes we're in agreement on this. There was little we could conclusively say about any domain until long term, concerted effort was made to understand each of them.

zanellato19 2y ago

Anecdotally, I have a friend that's doing bio-chem doctorate and she has said that replication rate on that is abysmal for biology, chemistry and consequently bio-chem. I'm sure some areas of physics have near 100% and some simply don't.