4 ms·

TL;DR: The hackjob happening in the social sciences will be used to launch ad hom attacks on much more rigorous scientific disciplines.

hirundo 5y ago
Optimistically, the hackjob happening in the social and other sciences will be used to launch a deep examination of epistemic practices. This does seem to have occurred in social sciences to some extent. There are no sciences so rigorous that they couldn't benefit from it.
- ContrarianBrit 5y ago
  Agreed. Kuhn is really worth reading - one of the greatest books of the 20th century.
ContrarianBrit 5y ago
But the hard sciences have also had problems with replication. It's worth reading the Structure of Scientific Revolutions by Thomas Kuhn; to some extent, science is socially embedded.
UncleMeat 5y ago
The social sciences are the only ones actually engaging in replication to detect the problems. The replication crisis in, say, software engineering research is not published but instead is implicitly understood by members of the community. In grad school I was told to generally assume that the tool with the second highest line in the key graph was the best tool. The top line was invariably the author's tool and its results rarely translated to other experiments. But the tool with the second line was usable by somebody other than its original author on a different problem space.
- throwawaygh 5y ago
  > software engineering research Empirical SE research is part of the social sciences. The subset of SE research that doesn't push hard for collaboration with social science departments is doing just fine.
  - UncleMeat 5y ago
    "I built a tool and it found this many bugs" doesn't feel like part of the social sciences, but these papers are regularly published in ICSE and FSE. I listed Software Engineering because it is one of the fields I know well but is way less esoteric than the other one I know well (PL). You'll find the same kinds of replication problems in POPL or OOPSLA papers too. I suspect that other subfields don't fare any better. I certainly hear from my ML friends that a huge portion of ML results are untrustworthy.
    - throwawaygh 5y ago
      > "I built a tool and it found this many bugs" doesn't feel like part of the social sciences, but these papers are regularly published in ICSE and FSE. Those papers have lots of issues, especially cherry-picking and narration of the results' significance. But the core claim -- "I found N bugs in the following codebases" -- is generally reproducible.
xyzzyz 5y ago
More rigorous disciplines should then explicitly and loudly disclaim any relationship with those fields driven by agenda and sloppy methods, and the universities should disown them and kick them out from under their roofs. It will not happen, though.
- throwawaygh 5y ago
  Scientists in the natural sciences/mathematics criticizing the social sciences (and also the humanities) is literally a trope. Natural scientists have been criticizing the issues in the social sciences for decades. In fact, a lot of the criticism that uncovered the replicated crisis originated in those communities.

hirundo 5y ago

Optimistically, the hackjob happening in the social and other sciences will be used to launch a deep examination of epistemic practices. This does seem to have occurred in social sciences to some extent. There are no sciences so rigorous that they couldn't benefit from it.

ContrarianBrit 5y ago
Agreed. Kuhn is really worth reading - one of the greatest books of the 20th century.

ContrarianBrit 5y ago

But the hard sciences have also had problems with replication. It's worth reading the Structure of Scientific Revolutions by Thomas Kuhn; to some extent, science is socially embedded.

UncleMeat 5y ago

The social sciences are the only ones actually engaging in replication to detect the problems. The replication crisis in, say, software engineering research is not published but instead is implicitly understood by members of the community. In grad school I was told to generally assume that the tool with the second highest line in the key graph was the best tool. The top line was invariably the author's tool and its results rarely translated to other experiments. But the tool with the second line was usable by somebody other than its original author on a different problem space.

throwawaygh 5y ago
> software engineering research Empirical SE research is part of the social sciences. The subset of SE research that doesn't push hard for collaboration with social science departments is doing just fine.
- UncleMeat 5y ago
  "I built a tool and it found this many bugs" doesn't feel like part of the social sciences, but these papers are regularly published in ICSE and FSE. I listed Software Engineering because it is one of the fields I know well but is way less esoteric than the other one I know well (PL). You'll find the same kinds of replication problems in POPL or OOPSLA papers too. I suspect that other subfields don't fare any better. I certainly hear from my ML friends that a huge portion of ML results are untrustworthy.
  - throwawaygh 5y ago
    > "I built a tool and it found this many bugs" doesn't feel like part of the social sciences, but these papers are regularly published in ICSE and FSE. Those papers have lots of issues, especially cherry-picking and narration of the results' significance. But the core claim -- "I found N bugs in the following codebases" -- is generally reproducible.

xyzzyz 5y ago

More rigorous disciplines should then explicitly and loudly disclaim any relationship with those fields driven by agenda and sloppy methods, and the universities should disown them and kick them out from under their roofs. It will not happen, though.

throwawaygh 5y ago
Scientists in the natural sciences/mathematics criticizing the social sciences (and also the humanities) is literally a trope. Natural scientists have been criticizing the issues in the social sciences for decades. In fact, a lot of the criticism that uncovered the replicated crisis originated in those communities.