3 ms·

The fact that so many papers are published rather works against your argument. The more papers are published in a given field, the more likely they are to be b

The fact that so many papers are published rather works against your argument. The more papers are published in a given field, the more likely they are to be bullshit. Of course, that applies for STEM fields as well as the social sciences. However, if we do comparisons across fields, such as looking at how many citations a typical paper in some field will receive, we find that STEM tends to do a lot better than the social sciences[1]. >Pick a field and take it down if you can. But trying to build a case against all of social science using the example of someone notoriously elevated despite a lackluster career as a publishing academic doesn't get you anywhere! The longer such fraud goes uncorrected, the more rewarded a person is for their fraud, and indeed the more lengths other people go to cover up such fraud (as was argued by the OP) the more likely such fraud is normalized in a particular field[2]. The Claudine Gay story suggests that, if anything, XKCD was wildly optimistic about the social sciences[3]. [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5446096/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5446096/ [2] https://danluu.com/wat/ https://danluu.com/wat/ [3] https://xkcd.com/451/ https://xkcd.com/451/

tptacek 3y ago
How does that follow? Is your premise that a static number of people are producing ever-increasing numbers of papers? That's not what's happening at all. There are lots of papers because there are lots of people doing research. As the number of vulnerability researchers increased, drastically in the 2000s, the number of vulnerabilities went up --- there was a point where people we tracking new vulnerabilities, across all platforms, by hand! But the median quality of a vulnerability research result didn't fall; in fact, while the variance increased, the peak quality of vulnerabilities improved.
- lliamander 3y ago
  I do not assume a static number of researchers, nor do I see why you would infer that from what I said. Of course there are more researchers going into these fields. The more people enter into a field, the more likely the typical person entering that field is going to be a mediocre researcher, and the more likely that their work is going to be crap. Sure, in the case of a very small and new field with promise of intellectually stimulating work and structural advantages for eliminating bullshit, adding more people and papers is likely going to be a net benefit. But realistically many established disciplines can only add people by lowering standards.
- lliamander 3y ago
  As an aside, why is this the thing you chose to focus on in responding to me? While I do think it is true and I am willing to defend it, it's not really the central issue. What I understand as the central point of your original comment was that we cannot condemn all of the social sciences (or 90% of the work in them) on the basis of the Claudine Gay story. I refute that with this sentence: >The longer such fraud goes uncorrected, the more rewarded a person is for their fraud, and indeed the more lengths other people go to cover up such fraud (as was argued by the OP) the more likely such fraud is normalized in a particular field. The Claudine Gay story suggests that, if anything, XKCD was wildly optimistic about the social sciences. Put another way, in the spirit of Scott Alexander's recent essay "Against Learning From Dramatic Events", the Claudine Gay story represents a significant deviation from what would be predicted by a model that assumed the social sciences had a more robust level of intellectual integrity.

tptacek 3y ago

How does that follow? Is your premise that a static number of people are producing ever-increasing numbers of papers? That's not what's happening at all. There are lots of papers because there are lots of people doing research. As the number of vulnerability researchers increased, drastically in the 2000s, the number of vulnerabilities went up --- there was a point where people we tracking new vulnerabilities, across all platforms, by hand! But the median quality of a vulnerability research result didn't fall; in fact, while the variance increased, the peak quality of vulnerabilities improved.

lliamander 3y ago
I do not assume a static number of researchers, nor do I see why you would infer that from what I said. Of course there are more researchers going into these fields. The more people enter into a field, the more likely the typical person entering that field is going to be a mediocre researcher, and the more likely that their work is going to be crap. Sure, in the case of a very small and new field with promise of intellectually stimulating work and structural advantages for eliminating bullshit, adding more people and papers is likely going to be a net benefit. But realistically many established disciplines can only add people by lowering standards.
lliamander 3y ago
As an aside, why is this the thing you chose to focus on in responding to me? While I do think it is true and I am willing to defend it, it's not really the central issue. What I understand as the central point of your original comment was that we cannot condemn all of the social sciences (or 90% of the work in them) on the basis of the Claudine Gay story. I refute that with this sentence: >The longer such fraud goes uncorrected, the more rewarded a person is for their fraud, and indeed the more lengths other people go to cover up such fraud (as was argued by the OP) the more likely such fraud is normalized in a particular field. The Claudine Gay story suggests that, if anything, XKCD was wildly optimistic about the social sciences. Put another way, in the spirit of Scott Alexander's recent essay "Against Learning From Dramatic Events", the Claudine Gay story represents a significant deviation from what would be predicted by a model that assumed the social sciences had a more robust level of intellectual integrity.