7 ms·

Stop treating non-sciences like sciences. The social sciences have a certain hypocrisy to them. "We do not need to be as grounded in math as the other sciences

by screye 3y ago

Stop treating non-sciences like sciences. The social sciences have a certain hypocrisy to them. "We do not need to be as grounded in math as the other sciences" runs up against "Our results are mathematically significant and deserve the same respect as the other sciences." Doing statistics in the hard sciences is easy, yet we study them at a graduate level. Setting up variable controls in hard sciences is easier, yet we spend years training people on eliminating even the smallest chance of error through rigid lab protocols. The hard sciences allow you to collect enough data-points, that a large effect size is almost always statistically significant... yet we report p-values religiously every step of the way. Social sciences have none of those affordances, yet they somehow get away with even less mathematical and statistical rigor. I mean this both in training and in practice. The social sciences are harder to control. It is a 'science' that can say very little about anything (purely due to difficulty of setting up controlled experiments). Yet, social science produces a disproportionate number of 'spicy' results. Not all fields will produce 'ground breaking' work and that should be fine. I'm not sure where the incentive/impetus for it comes from, but social scientists build storied careers on results that would get rejected from any respected science journal for lack of rigor. It is as if there is this pressure on every academic to prove something fundamental about human nature few years, all while it's plainly clear that we as humans know almost nothing about human nature. If centuries of work in this field has not been able to find a few foundational facts (fancy alliteration), then maybe the field needs to reconsider its ambitiousness and self-assured confidence.

_yb2s 3y ago
I’ve found the opposite… because of the complexity and noise the social sciences are using more sophisticated statistical methods than the “hard” sciences. In grad school I took Bayesian probability theory classes from social scientists because my “hard” science peers were still using frequentist statistical tests, often applied inappropriately by just copying methods from past papers that also applied the methods inappropriately. The social sciences in general seem to be a decade or two ahead in adopting improved methods, and researchers spend a lot more time understanding these methods, writing custom software and models, etc. IMO, mathematical and computational “literacy” is particularly low in the medical and life sciences compared to the social sciences, physics and chemistry are not quite as bad.
- TheMagicHorsey 3y ago
  I think it really varies department to department. Was your program in an American university?
  - _yb2s 3y ago
    An R1 research university in the USA Look at the most modern open source R and Python packages for statistical inference… probably 90 percent are developed and maintained by social science research labs
- gowld 3y ago
  Indeed, social sciences use the most sophisticated alchemy to product "knowledge" from ignorance. :-) Bayesian statistics is great, you can get any posterior you want by choosing good priors.
  - wizzwizz4 3y ago
    But papers don't publish priors. They publish evidence. "This study provides 5dB of evidence that the pigs do not reactionless flight capabilities, and 300dB of evidence that I don't know how physics works. However, according to my priors, the flying pigs hypothesis still comes out ahead." said no study ever.
    - mike_hearn 3y ago
      In reality a lot of them publish priors dressed up to look like evidence. As an example see Flaxman2020. It's best to model journals as magazines that publish anything that vaguely resembles science (has equations, charts, citations, claims of data). It doesn't have to actually be science. If you want to get a paper published it's totally normal to write 10,000 lines of C or FORTRAN, claim it models some natural process, publish a paper with the graphs it outputs and call it a day. The code isn't available. The input data isn't available. The assumptions aren't documented properly or at all. And the outputs are unvalidated, therefore not even laying the foundations of being evidence of anything. Doesn't matter, it'll get published, in some fields this is the vast majority of what gets published (e.g. epidemiology).
currymj 3y ago
Social science is not the problem. Biology is definitely a "real" science yet the problems with fraud in biology are, if anything, worse. See the recent cases involving fabricated data in a Nature paper about Alzheimer's. https://www.science.org/content/article/potential-fabrication-research-images-threatens-key-theory-alzheimers-disease https://www.science.org/content/article/potential-fabricatio... That actually seems to be a common pattern, there's another case now with strong evidence of data fraud in multiple Science/Nature papers, possibly implicating the president of Stanford. https://stanforddaily.com/2023/04/25/stanford-president-dodges-research-misconduct-questions/ https://stanforddaily.com/2023/04/25/stanford-president-dodg... The problems seem less severe in physics, but there was that recent case where supposed experimental evidence of majorana fermions had to be retracted, with some research practices that look dubious in retrospect. Or the famous Schön scandal at Bell Labs in semiconductor physics, where there was outright fraud. https://en.wikipedia.org/wiki/Sch%C3%B6n_scandal https://en.wikipedia.org/wiki/Sch%C3%B6n_scandal
- justinclift 3y ago
  > Social science is not the problem. Could it be there's more than one problem? :)
  - gowld 3y ago
    Broadly, the problem is replication. Social science is non-replicable because the parameters are too fuzzy. (People are more diverse than electrons). Lab science is non-replicable because the experiments are too expensive to set up, and sometimes are intentionally kept as trade secrets for economic reasons.
    - cameldrv 3y ago
      Equally broadly, there's huge pressure to report positive results. If the incentive structure were even tilted a bit to make reporting negative or ambiguous results less career-damaging, we would see much less of this. I don't think most scientists want to fudge the truth, but in many cases they feel that they have no choice. The pressure in most of these sciences is to consistently get a paper for every expensive experiment that is done, but if you're doing that, and published work has to be a positive result, you're not really doing science. You could just publish your hypothesis and be done with it if you already know the result of the experiment will be ginned up in some way to be positive.
taeric 3y ago
I'll push that it is also "stop treating academia as such high stakes as we treat it." We have "raised the bar" so much that you are basically thrown out with failure. If you make it life or death stakes, expect people to fight appropriately. At some level, that will be pushing the data harder than it has a right to be pushed.
- vacuity 3y ago
  The classic: “It is difficult to get a man to understand something, when his salary depends on his not understanding it.” - Upton Sinclair Except possibly even more high stakes than that, since academics' very reputations are on the line.
  - ohnonononohno 3y ago
    REM: sunk-cost-falacy https://en.m.wikipedia.org/wiki/Hat_puzzle https://en.m.wikipedia.org/wiki/Hat_puzzle Gödel's incompleteness theorem (philosophical); 'Theorem states that no consistent system of axioms can be listed by an effective procedure, capable of proving all truth' (-; https://en.m.wikipedia.org/wiki/Epistemic_logic https://en.m.wikipedia.org/wiki/Epistemic_logic Hint: HN in 2016 regards...
- fullshark 3y ago
  The only way to do this is to admit fewer PhD students, which probably should happen. There's only so many professorships, and some will always be more coveted than others, leading to those stakes.
  - taeric 3y ago
    I could posit a few other ways. Could also make it so that we aren't financially dependent on wild success, for example.
  - pas 3y ago
    It's probably not the absolute number of PhD students that's the problem. It's the crushing despair of uncertainty, the publish-and-eventually-still-perish incinerator, the cult of positive results, the completely ridiculous inefficiency of the actual work (because it's done mostly by inexperienced grad students, and by the time they know what's up they are already off to some godforsaken postdoc or worse, or leave academia for good if they are lucky). Not to mention the whole tenure system is questionable. Yes, it's good that at least a few percent of academia has job security, but the selection process is full of perverse incentives and arguably society could get better results if tenure was for 10 years and based on a lottery.
mc32 3y ago
Yah they should be called Studies. Social studies, psychologic studies, economic studies, etc.
- gowld 3y ago
  mathematical studies, chemical studies, biological studies. Or we could use fancy Latin suffixes: sociology, psychology, economicology, biology, chemiology, pharmacology, mathology (https://www.youtube.com/channel/UC1_uAIS3r8Vu6JjXWvastJg https://www.youtube.com/channel/UC1_uAIS3r8Vu6JjXWvastJg).
  - mc32 3y ago
    science implies a rigor which does not exist in those studies. Quite a few cities, NGOs, non-profits and other organizations make decisions on quick and dirty studies by university students involving 30 or 40 subjects because the results "feel good". The evening news will also trumpet these studies "A new study out of the University of X reveals that..." Politicians, "Well, there is a new study from the University of X that says if we..."
    - bakuninsbart 3y ago
      What does science mean then? And what kind of rigor are you meaning specifically?
tejohnso 3y ago
> social scientists build storied careers on results that would get rejected from any respected science journal for lack of rigor Wasn't it called social studies? When did social studies get branded as science in the first place?
- nemo44x 3y ago
  They started using some tools that scientists use and decided they too were a "Science" (because we used a tool that actual science uses) because it was the best way to be taken as seriously as real science. There has been a brewing jealousy in the academy over many decades with the type of success hard sciences have seen. So, just put the word "science" in your title and pretend you're doing "Science" and now your opinions get to be "the science" and if someone scrutinizes it (like actual science would be and is open to) you can say "the science is settled." Of course the best irony is the current fad of rejecting "objective truth" (on whatever grounds are useful - it's racist, etc) and yet still claiming you are doing a science. Not the search for truth but rather the search for your truth.
- bakuninsbart 3y ago
  It was called social science first, then rebranded as social studies later to be more inclusive, after the British society of the sciences made the term 'science' a lot more exclusive. https://www.cambridge.org/core/journals/modern-intellectual-history/article/abs/three-versions-of-social-science-in-late-eighteenthcentury-france/EC816C9BED4204BD5CB58DCB3EB2382B https://www.cambridge.org/core/journals/modern-intellectual-...
IKantRead 3y ago
The problem is funding. The best psychology work tends to be much closer to philosophy than science. But philosophy generally doesn't get much funding. Anyone interested in psychology will ultimately be forced to optimize for funding, which inevitably leads to fraudulent work since psychology based on scientific principles simply isn't possible yet as we don't know nearly enough about the function of the mind to explore the areas psychology is interested in.
AndyMcConachie 3y ago
You're argument is basically, "It's really hard so we shouldn't try." A good social scientist understands everything you're pointing out. This isn't news to anyone nor is it even really controversial. But we can witness phenomena in society so people try and understand them. Would you prefer we not try and understand them? Just give up because it's too hard to figure out?
- AlexandrB 3y ago
  There's definitely an appeal to solving a simpler, technological problem instead of tackling the more difficult, societal one. Consider crypto, which tries to solve concentration of power in financial institutions by creating a distributed technology for tracking financial transactions. Unfortunately this just kicks the can down the road.
  - User23 3y ago
    Sometimes the can just needs to be kicked indefinitely. Every time you perform some kind of maintenance on something that degrades over time, like a house or a car, you're can kicking, because unless the thing being maintained ceases to exist, you or your successors will eventually face the same issue again and again. Frankly it's a pretty lousy metaphor.
mistermann 3y ago
> Stop treating non-sciences like sciences. Stop ignoring the important nuances of the different problem spaces science works in, and lacks adequate sophistication in its methodologies for dealing with those nuances resulting in these gong shows regularly happening and then explained away as if there's no fundamental problem.
rossdavidh 3y ago
Thus far, I am unaware of any field of science which has seriously looked for problems like these, that has not found it.
chengiz 3y ago
This is just your run of the mill STEM looking down upon other fields argument. There is rampant fraud in STEM. If anything it's harded to detect because the people who do it are better informed on how to hide it.
bjourne 3y ago
Fabricated data and fraudulent results are not unknown to the hard sciences either. See f.e https://www.youtube.com/watch?v=nfDoml-Db64 https://www.youtube.com/watch?v=nfDoml-Db64 A professor manipulating data for an amusing pop-psychology result is small potatoes in comparison to what has rocked the physics world.

by screye 3y ago

_yb2s 3y ago

I’ve found the opposite… because of the complexity and noise the social sciences are using more sophisticated statistical methods than the “hard” sciences. In grad school I took Bayesian probability theory classes from social scientists because my “hard” science peers were still using frequentist statistical tests, often applied inappropriately by just copying methods from past papers that also applied the methods inappropriately. The social sciences in general seem to be a decade or two ahead in adopting improved methods, and researchers spend a lot more time understanding these methods, writing custom software and models, etc. IMO, mathematical and computational “literacy” is particularly low in the medical and life sciences compared to the social sciences, physics and chemistry are not quite as bad.

TheMagicHorsey 3y ago
I think it really varies department to department. Was your program in an American university?
- _yb2s 3y ago
  An R1 research university in the USA Look at the most modern open source R and Python packages for statistical inference… probably 90 percent are developed and maintained by social science research labs
gowld 3y ago
Indeed, social sciences use the most sophisticated alchemy to product "knowledge" from ignorance. :-) Bayesian statistics is great, you can get any posterior you want by choosing good priors.
- wizzwizz4 3y ago
  But papers don't publish priors. They publish evidence. "This study provides 5dB of evidence that the pigs do not reactionless flight capabilities, and 300dB of evidence that I don't know how physics works. However, according to my priors, the flying pigs hypothesis still comes out ahead." said no study ever.
  - mike_hearn 3y ago
    In reality a lot of them publish priors dressed up to look like evidence. As an example see Flaxman2020. It's best to model journals as magazines that publish anything that vaguely resembles science (has equations, charts, citations, claims of data). It doesn't have to actually be science. If you want to get a paper published it's totally normal to write 10,000 lines of C or FORTRAN, claim it models some natural process, publish a paper with the graphs it outputs and call it a day. The code isn't available. The input data isn't available. The assumptions aren't documented properly or at all. And the outputs are unvalidated, therefore not even laying the foundations of being evidence of anything. Doesn't matter, it'll get published, in some fields this is the vast majority of what gets published (e.g. epidemiology).

currymj 3y ago

Social science is not the problem. Biology is definitely a "real" science yet the problems with fraud in biology are, if anything, worse. See the recent cases involving fabricated data in a Nature paper about Alzheimer's. https://www.science.org/content/article/potential-fabrication-research-images-threatens-key-theory-alzheimers-disease https://www.science.org/content/article/potential-fabricatio... That actually seems to be a common pattern, there's another case now with strong evidence of data fraud in multiple Science/Nature papers, possibly implicating the president of Stanford. https://stanforddaily.com/2023/04/25/stanford-president-dodges-research-misconduct-questions/ https://stanforddaily.com/2023/04/25/stanford-president-dodg... The problems seem less severe in physics, but there was that recent case where supposed experimental evidence of majorana fermions had to be retracted, with some research practices that look dubious in retrospect. Or the famous Schön scandal at Bell Labs in semiconductor physics, where there was outright fraud. https://en.wikipedia.org/wiki/Sch%C3%B6n_scandal https://en.wikipedia.org/wiki/Sch%C3%B6n_scandal

justinclift 3y ago
> Social science is not the problem. Could it be there's more than one problem? :)
- gowld 3y ago
  Broadly, the problem is replication. Social science is non-replicable because the parameters are too fuzzy. (People are more diverse than electrons). Lab science is non-replicable because the experiments are too expensive to set up, and sometimes are intentionally kept as trade secrets for economic reasons.
  - cameldrv 3y ago
    Equally broadly, there's huge pressure to report positive results. If the incentive structure were even tilted a bit to make reporting negative or ambiguous results less career-damaging, we would see much less of this. I don't think most scientists want to fudge the truth, but in many cases they feel that they have no choice. The pressure in most of these sciences is to consistently get a paper for every expensive experiment that is done, but if you're doing that, and published work has to be a positive result, you're not really doing science. You could just publish your hypothesis and be done with it if you already know the result of the experiment will be ginned up in some way to be positive.

taeric 3y ago

I'll push that it is also "stop treating academia as such high stakes as we treat it." We have "raised the bar" so much that you are basically thrown out with failure. If you make it life or death stakes, expect people to fight appropriately. At some level, that will be pushing the data harder than it has a right to be pushed.

vacuity 3y ago
The classic: “It is difficult to get a man to understand something, when his salary depends on his not understanding it.” - Upton Sinclair Except possibly even more high stakes than that, since academics' very reputations are on the line.
- ohnonononohno 3y ago
  REM: sunk-cost-falacy https://en.m.wikipedia.org/wiki/Hat_puzzle https://en.m.wikipedia.org/wiki/Hat_puzzle Gödel's incompleteness theorem (philosophical); 'Theorem states that no consistent system of axioms can be listed by an effective procedure, capable of proving all truth' (-; https://en.m.wikipedia.org/wiki/Epistemic_logic https://en.m.wikipedia.org/wiki/Epistemic_logic Hint: HN in 2016 regards...
fullshark 3y ago
The only way to do this is to admit fewer PhD students, which probably should happen. There's only so many professorships, and some will always be more coveted than others, leading to those stakes.
- taeric 3y ago
  I could posit a few other ways. Could also make it so that we aren't financially dependent on wild success, for example.
- pas 3y ago
  It's probably not the absolute number of PhD students that's the problem. It's the crushing despair of uncertainty, the publish-and-eventually-still-perish incinerator, the cult of positive results, the completely ridiculous inefficiency of the actual work (because it's done mostly by inexperienced grad students, and by the time they know what's up they are already off to some godforsaken postdoc or worse, or leave academia for good if they are lucky). Not to mention the whole tenure system is questionable. Yes, it's good that at least a few percent of academia has job security, but the selection process is full of perverse incentives and arguably society could get better results if tenure was for 10 years and based on a lottery.

mc32 3y ago

Yah they should be called Studies. Social studies, psychologic studies, economic studies, etc.

gowld 3y ago
mathematical studies, chemical studies, biological studies. Or we could use fancy Latin suffixes: sociology, psychology, economicology, biology, chemiology, pharmacology, mathology (https://www.youtube.com/channel/UC1_uAIS3r8Vu6JjXWvastJg https://www.youtube.com/channel/UC1_uAIS3r8Vu6JjXWvastJg).
- mc32 3y ago
  science implies a rigor which does not exist in those studies. Quite a few cities, NGOs, non-profits and other organizations make decisions on quick and dirty studies by university students involving 30 or 40 subjects because the results "feel good". The evening news will also trumpet these studies "A new study out of the University of X reveals that..." Politicians, "Well, there is a new study from the University of X that says if we..."
  - bakuninsbart 3y ago
    What does science mean then? And what kind of rigor are you meaning specifically?

tejohnso 3y ago

> social scientists build storied careers on results that would get rejected from any respected science journal for lack of rigor Wasn't it called social studies? When did social studies get branded as science in the first place?

nemo44x 3y ago
They started using some tools that scientists use and decided they too were a "Science" (because we used a tool that actual science uses) because it was the best way to be taken as seriously as real science. There has been a brewing jealousy in the academy over many decades with the type of success hard sciences have seen. So, just put the word "science" in your title and pretend you're doing "Science" and now your opinions get to be "the science" and if someone scrutinizes it (like actual science would be and is open to) you can say "the science is settled." Of course the best irony is the current fad of rejecting "objective truth" (on whatever grounds are useful - it's racist, etc) and yet still claiming you are doing a science. Not the search for truth but rather the search for your truth.
bakuninsbart 3y ago
It was called social science first, then rebranded as social studies later to be more inclusive, after the British society of the sciences made the term 'science' a lot more exclusive. https://www.cambridge.org/core/journals/modern-intellectual-history/article/abs/three-versions-of-social-science-in-late-eighteenthcentury-france/EC816C9BED4204BD5CB58DCB3EB2382B https://www.cambridge.org/core/journals/modern-intellectual-...

IKantRead 3y ago

The problem is funding. The best psychology work tends to be much closer to philosophy than science. But philosophy generally doesn't get much funding. Anyone interested in psychology will ultimately be forced to optimize for funding, which inevitably leads to fraudulent work since psychology based on scientific principles simply isn't possible yet as we don't know nearly enough about the function of the mind to explore the areas psychology is interested in.

AndyMcConachie 3y ago

You're argument is basically, "It's really hard so we shouldn't try." A good social scientist understands everything you're pointing out. This isn't news to anyone nor is it even really controversial. But we can witness phenomena in society so people try and understand them. Would you prefer we not try and understand them? Just give up because it's too hard to figure out?

AlexandrB 3y ago
There's definitely an appeal to solving a simpler, technological problem instead of tackling the more difficult, societal one. Consider crypto, which tries to solve concentration of power in financial institutions by creating a distributed technology for tracking financial transactions. Unfortunately this just kicks the can down the road.
- User23 3y ago
  Sometimes the can just needs to be kicked indefinitely. Every time you perform some kind of maintenance on something that degrades over time, like a house or a car, you're can kicking, because unless the thing being maintained ceases to exist, you or your successors will eventually face the same issue again and again. Frankly it's a pretty lousy metaphor.

mistermann 3y ago

> Stop treating non-sciences like sciences. Stop ignoring the important nuances of the different problem spaces science works in, and lacks adequate sophistication in its methodologies for dealing with those nuances resulting in these gong shows regularly happening and then explained away as if there's no fundamental problem.

rossdavidh 3y ago

Thus far, I am unaware of any field of science which has seriously looked for problems like these, that has not found it.

chengiz 3y ago

This is just your run of the mill STEM looking down upon other fields argument. There is rampant fraud in STEM. If anything it's harded to detect because the people who do it are better informed on how to hide it.

bjourne 3y ago

Fabricated data and fraudulent results are not unknown to the hard sciences either. See f.e https://www.youtube.com/watch?v=nfDoml-Db64 https://www.youtube.com/watch?v=nfDoml-Db64 A professor manipulating data for an amusing pop-psychology result is small potatoes in comparison to what has rocked the physics world.