3 ms·

Interesting that this is actually a study. I see the major problems here (shaky citation sources, unfounded claims, poor data), but I also see the difficulty in

Interesting that this is actually a study. I see the major problems here (shaky citation sources, unfounded claims, poor data), but I also see the difficulty in attempting a study like this. I can agree with the claims even if I recognize the problems. Still, having Reddit as your only source of data seems like a recipe for disaster. Maybe a better title would have been "examining incel lexicon and dissemination into popular discourse" - but then it would have had to be a better study.

dekhn 5y ago
One of the things that all good grad students learn is how to do paper triage. Presume that you are faced with a large number of papers and a limited time to read a subset of them. How do you allocate your time to papers? I, and many others follow the "3 bad attributes" rule. If you're reading the paper and you see 3 shaky things- small sample size, multiple tests without correction, concluding something not supported by the data- you just stop reading the paper. The other detail is that you should absolutely never start by looking at figures. Figures lie. They are designed to convince you what the author wants you to believe. Instead, you need to read the parts of the paper that describe what they did in detail, and build a model of that (this is why when authors share enough code and data to reproduce their experiment fully, people like me are happy because that saves literally a day or two of work), and convince yourself that you believe their work was done competently. Most papers were not done competently. Most of what you're reading is people who managed to convince themselves they found significance instead of null hypothesis. At the end you're typically left with a very small number of good papers which pass every test you apply to them: they state what they did (materials and methods), what they got (results). You should care less about citations as long as the authors don't make obvious mistakes that are well-esconced in the literature). Then, and only then do you pay attention. I rejected this paper within a few sentences because it looks to be an activist paper wrapped in semi-quantative analysis on a low quality dataset.

dekhn 5y ago

One of the things that all good grad students learn is how to do paper triage. Presume that you are faced with a large number of papers and a limited time to read a subset of them. How do you allocate your time to papers? I, and many others follow the "3 bad attributes" rule. If you're reading the paper and you see 3 shaky things- small sample size, multiple tests without correction, concluding something not supported by the data- you just stop reading the paper. The other detail is that you should absolutely never start by looking at figures. Figures lie. They are designed to convince you what the author wants you to believe. Instead, you need to read the parts of the paper that describe what they did in detail, and build a model of that (this is why when authors share enough code and data to reproduce their experiment fully, people like me are happy because that saves literally a day or two of work), and convince yourself that you believe their work was done competently. Most papers were not done competently. Most of what you're reading is people who managed to convince themselves they found significance instead of null hypothesis. At the end you're typically left with a very small number of good papers which pass every test you apply to them: they state what they did (materials and methods), what they got (results). You should care less about citations as long as the authors don't make obvious mistakes that are well-esconced in the literature). Then, and only then do you pay attention. I rejected this paper within a few sentences because it looks to be an activist paper wrapped in semi-quantative analysis on a low quality dataset.