6 ms·

Social science is not the problem. Biology is definitely a "real" science yet the problems with fraud in biology are, if anything, worse. See the recent cases

Social science is not the problem. Biology is definitely a "real" science yet the problems with fraud in biology are, if anything, worse. See the recent cases involving fabricated data in a Nature paper about Alzheimer's. https://www.science.org/content/article/potential-fabrication-research-images-threatens-key-theory-alzheimers-disease https://www.science.org/content/article/potential-fabricatio... That actually seems to be a common pattern, there's another case now with strong evidence of data fraud in multiple Science/Nature papers, possibly implicating the president of Stanford. https://stanforddaily.com/2023/04/25/stanford-president-dodges-research-misconduct-questions/ https://stanforddaily.com/2023/04/25/stanford-president-dodg... The problems seem less severe in physics, but there was that recent case where supposed experimental evidence of majorana fermions had to be retracted, with some research practices that look dubious in retrospect. Or the famous Schön scandal at Bell Labs in semiconductor physics, where there was outright fraud. https://en.wikipedia.org/wiki/Sch%C3%B6n_scandal https://en.wikipedia.org/wiki/Sch%C3%B6n_scandal

justinclift 3y ago
> Social science is not the problem. Could it be there's more than one problem? :)
- gowld 3y ago
  Broadly, the problem is replication. Social science is non-replicable because the parameters are too fuzzy. (People are more diverse than electrons). Lab science is non-replicable because the experiments are too expensive to set up, and sometimes are intentionally kept as trade secrets for economic reasons.
  - cameldrv 3y ago
    Equally broadly, there's huge pressure to report positive results. If the incentive structure were even tilted a bit to make reporting negative or ambiguous results less career-damaging, we would see much less of this. I don't think most scientists want to fudge the truth, but in many cases they feel that they have no choice. The pressure in most of these sciences is to consistently get a paper for every expensive experiment that is done, but if you're doing that, and published work has to be a positive result, you're not really doing science. You could just publish your hypothesis and be done with it if you already know the result of the experiment will be ginned up in some way to be positive.
    - chongli 3y ago
      If the incentive structure were even tilted a bit to make reporting negative or ambiguous results less career-damaging, we would see much less of this I don't see how we do this. Ultimately, the buck stops with the public. Governments fund the sciences because the public demands it. If scientists spend more time reporting negative or ambiguous results then the public begin to question what they're paying for. People want breakthroughs and scientific revolutions, not "business as usual." And this bias is not limited to the public, scientists themselves want to see new science.
      - mistermann 3y ago
        > Governments fund the sciences because the public demands it. Is this a scientific fact? Secondly, is it an epistemically flawless fact (which is similar, but also different)? In what way exactly does "the public" "demand" that which science does?
        chongli 3y ago
        Is this a scientific fact? Secondly, is it an epistemically flawless fact (which is similar, but also different)? Why don’t you check on Google Scholar and report back to me? In what way exactly does "the public" "demand" that which science does? In the economic way that we say most things are demanded. I’m not saying that people are going to be out in the streets protesting if scientists start publishing more negative results. I’m saying that the general public attitude towards science (the one measured in polls) is affected by the public’s perception that scientists are producing important and meaningful results.
        mistermann 3y ago
        > Why don’t you check on Google Scholar and report back to me? Standard conventions for burden of proof I suppose. Surely you don't actually have no evidence to back up your claim, do you? > In the economic way that we say most things are demanded. Are you discerning support for science from economic data of some sort? >I’m saying that the general public attitude towards science (the one measured in polls) is affected by the public’s perception that scientists are producing important and meaningful results. And might it be possible that the public's perception of science could be affected by all of the free marketing it's gotten in the last five or so years?
  - godelski 3y ago
    > because the parameters are too fuzzy. This just advocates for a system that more accurately captures uncertainty in the models. Which, would technically make the results replicable but reduce their impact. At least in the short term. Impact in the long term is harder to determine. Though, I really think A LOT of fields need to be far more honest about their levels of uncertainty and any ones that aren't (especially empirically based ones) are bordering snake oil territory.
  - JumpCrisscross 3y ago
    > Social science is non-replicable because the parameters are too fuzzy. (People are more diverse than electrons) Social scientists tolerate their peers who see themselves as agents of change more than as scientists. A Stanford education specialist, who wrote California's new mathematics standards for public schools, framed--among other things--a well-reasoned takedown of one of her papers [1] as personal attacks [2]. Consider the same happening in physics. Paper published. Critique follows. Is there any world where a physicist maintains respectability by ignoring the content of the critique while claiming harassment? Yet that's not only tolerated in many social sciences, it's the norm. Add to that lucrative consulting gigs, and researchers have a perverse incentive to (a) extrapolate N=2 studies of hung-over undergrads to corporate America and geopolitics and (b) personally attack anyone who dares challenge their premise. [1] https://www.nonpartisaneducation.org/Review/Articles/v8n1.pdf https://www.nonpartisaneducation.org/Review/Articles/v8n1.pd... [2] https://joboaler.people.stanford.edu/sites/g/files/sbiybj28666/files/media/file/response-to-3-areas-3.pdf https://joboaler.people.stanford.edu/sites/g/files/sbiybj286...
  - crazygringo 3y ago
    > Social science is non-replicable because the parameters are too fuzzy. (People are more diverse than electrons). This isn't true at all. It's quite easy to create well-defined survey questions and experiments. It's entirely replicable. The main problems with social sciences are that they have historically been perfectly fine publishing studies based on statistically tiny numbers of participants, as well as assuming findings from tiny groups such as university undergrad test subjects could be extrapolated to all of humanity. These problems are eminently solvable with stricter statistical standards, and much greater cross-cultural research. It's just research becomes that much more expensive.
User23 3y ago
Biology is a huge field. Some parts of it are rigorous hard science, like a good chunk of cellular biology. Other parts of it are pure speculation, like the origin of life. And some are obscure and confused, like what exactly is life anyhow? As an example for that last one biologists appear to have settled for begging the question and saying well a thing is alive if it has life processes. I don't know that anyone else does, but I like to draw a distinction between experimental scientific inquiry and rigorously observed phenomenology. Properly speaking I say a great deal of what's called science is really a species of phenomenology. For example cosmology is pure phenomenology. We can't actually run experiments on what happens when you smash two black holes together. However we can rigorously observe the phenomenon and abduce a descriptive model that can potentially even make predictions. But the word "science" has become a totem, so there's a lot of cargo culting[1] and punishing blasphemers. However, I don't really think that whether a field is science, phenomenology, or some blend of both has much effect on the rate of fraud. Rather, I hypothesize the dominant factor in the incidence of fraud is the perceived magnitude of the potential benefit to be obtained. The Alzheimer's fraud is a good example of that. There are hundreds of billions and maybe even trillions of dollars in play in that space. Drug companies have already poured billions down the drain chasing dead ends so anyone who can sell them hope can potentially get a substantial payday. An amusing corollary of this is that given the relatively small stakes social science fraudsters are playing for, we can conclude that they are small-minded in the sense that they aren't thinking nearly as big as the drug company fraudsters. [1] https://calteches.library.caltech.edu/51/2/CargoCult.htm https://calteches.library.caltech.edu/51/2/CargoCult.htm
droptablemain 3y ago
There are certainly problems in academia outside of social sciences but nothing quite like this: https://en.wikipedia.org/wiki/Grievance_studies_affair https://en.wikipedia.org/wiki/Grievance_studies_affair

justinclift 3y ago

> Social science is not the problem. Could it be there's more than one problem? :)

gowld 3y ago
Broadly, the problem is replication. Social science is non-replicable because the parameters are too fuzzy. (People are more diverse than electrons). Lab science is non-replicable because the experiments are too expensive to set up, and sometimes are intentionally kept as trade secrets for economic reasons.
- cameldrv 3y ago
  Equally broadly, there's huge pressure to report positive results. If the incentive structure were even tilted a bit to make reporting negative or ambiguous results less career-damaging, we would see much less of this. I don't think most scientists want to fudge the truth, but in many cases they feel that they have no choice. The pressure in most of these sciences is to consistently get a paper for every expensive experiment that is done, but if you're doing that, and published work has to be a positive result, you're not really doing science. You could just publish your hypothesis and be done with it if you already know the result of the experiment will be ginned up in some way to be positive.
  - chongli 3y ago
    If the incentive structure were even tilted a bit to make reporting negative or ambiguous results less career-damaging, we would see much less of this I don't see how we do this. Ultimately, the buck stops with the public. Governments fund the sciences because the public demands it. If scientists spend more time reporting negative or ambiguous results then the public begin to question what they're paying for. People want breakthroughs and scientific revolutions, not "business as usual." And this bias is not limited to the public, scientists themselves want to see new science.
    - mistermann 3y ago
      > Governments fund the sciences because the public demands it. Is this a scientific fact? Secondly, is it an epistemically flawless fact (which is similar, but also different)? In what way exactly does "the public" "demand" that which science does?
      - chongli 3y ago
        Is this a scientific fact? Secondly, is it an epistemically flawless fact (which is similar, but also different)? Why don’t you check on Google Scholar and report back to me? In what way exactly does "the public" "demand" that which science does? In the economic way that we say most things are demanded. I’m not saying that people are going to be out in the streets protesting if scientists start publishing more negative results. I’m saying that the general public attitude towards science (the one measured in polls) is affected by the public’s perception that scientists are producing important and meaningful results.
        mistermann 3y ago
        > Why don’t you check on Google Scholar and report back to me? Standard conventions for burden of proof I suppose. Surely you don't actually have no evidence to back up your claim, do you? > In the economic way that we say most things are demanded. Are you discerning support for science from economic data of some sort? >I’m saying that the general public attitude towards science (the one measured in polls) is affected by the public’s perception that scientists are producing important and meaningful results. And might it be possible that the public's perception of science could be affected by all of the free marketing it's gotten in the last five or so years?
- godelski 3y ago
  > because the parameters are too fuzzy. This just advocates for a system that more accurately captures uncertainty in the models. Which, would technically make the results replicable but reduce their impact. At least in the short term. Impact in the long term is harder to determine. Though, I really think A LOT of fields need to be far more honest about their levels of uncertainty and any ones that aren't (especially empirically based ones) are bordering snake oil territory.
- JumpCrisscross 3y ago
  > Social science is non-replicable because the parameters are too fuzzy. (People are more diverse than electrons) Social scientists tolerate their peers who see themselves as agents of change more than as scientists. A Stanford education specialist, who wrote California's new mathematics standards for public schools, framed--among other things--a well-reasoned takedown of one of her papers [1] as personal attacks [2]. Consider the same happening in physics. Paper published. Critique follows. Is there any world where a physicist maintains respectability by ignoring the content of the critique while claiming harassment? Yet that's not only tolerated in many social sciences, it's the norm. Add to that lucrative consulting gigs, and researchers have a perverse incentive to (a) extrapolate N=2 studies of hung-over undergrads to corporate America and geopolitics and (b) personally attack anyone who dares challenge their premise. [1] https://www.nonpartisaneducation.org/Review/Articles/v8n1.pdf https://www.nonpartisaneducation.org/Review/Articles/v8n1.pd... [2] https://joboaler.people.stanford.edu/sites/g/files/sbiybj28666/files/media/file/response-to-3-areas-3.pdf https://joboaler.people.stanford.edu/sites/g/files/sbiybj286...
- crazygringo 3y ago
  > Social science is non-replicable because the parameters are too fuzzy. (People are more diverse than electrons). This isn't true at all. It's quite easy to create well-defined survey questions and experiments. It's entirely replicable. The main problems with social sciences are that they have historically been perfectly fine publishing studies based on statistically tiny numbers of participants, as well as assuming findings from tiny groups such as university undergrad test subjects could be extrapolated to all of humanity. These problems are eminently solvable with stricter statistical standards, and much greater cross-cultural research. It's just research becomes that much more expensive.

User23 3y ago

Biology is a huge field. Some parts of it are rigorous hard science, like a good chunk of cellular biology. Other parts of it are pure speculation, like the origin of life. And some are obscure and confused, like what exactly is life anyhow? As an example for that last one biologists appear to have settled for begging the question and saying well a thing is alive if it has life processes. I don't know that anyone else does, but I like to draw a distinction between experimental scientific inquiry and rigorously observed phenomenology. Properly speaking I say a great deal of what's called science is really a species of phenomenology. For example cosmology is pure phenomenology. We can't actually run experiments on what happens when you smash two black holes together. However we can rigorously observe the phenomenon and abduce a descriptive model that can potentially even make predictions. But the word "science" has become a totem, so there's a lot of cargo culting[1] and punishing blasphemers. However, I don't really think that whether a field is science, phenomenology, or some blend of both has much effect on the rate of fraud. Rather, I hypothesize the dominant factor in the incidence of fraud is the perceived magnitude of the potential benefit to be obtained. The Alzheimer's fraud is a good example of that. There are hundreds of billions and maybe even trillions of dollars in play in that space. Drug companies have already poured billions down the drain chasing dead ends so anyone who can sell them hope can potentially get a substantial payday. An amusing corollary of this is that given the relatively small stakes social science fraudsters are playing for, we can conclude that they are small-minded in the sense that they aren't thinking nearly as big as the drug company fraudsters. [1] https://calteches.library.caltech.edu/51/2/CargoCult.htm https://calteches.library.caltech.edu/51/2/CargoCult.htm

droptablemain 3y ago

There are certainly problems in academia outside of social sciences but nothing quite like this: https://en.wikipedia.org/wiki/Grievance_studies_affair https://en.wikipedia.org/wiki/Grievance_studies_affair