6 ms·
The statistical analysis in this paper is really poor. Without the raw data I would say it is impossible to tell anything from this study. > change in citation
by barbegal 6y ago
The statistical analysis in this paper is really poor. Without the raw data I would say it is impossible to tell anything from this study.
> change in citations at 1 year (Tweeted +3.1±2.4 vs. Non-Tweeted +0.7±1.3,
p<0.001)
I don't know what that p<0.001 means in this context but there is certainly more than a 0.1% chance that the null hypothesis (the two distributions are the same) given those 95% confidence intervals.
The graphs that are used to illustrate the paper have completely different confidence intervals than the numbers in the paper and look too good to be true.
Given the average tweet was engaged with less than 16 times I really doubt that the effect size would be this big. This looks to me to be the case that a couple of good papers happened to be tweeted and that nothing statistically significant can be gleaned from this paper.
Having said that I am not a professional statistician so I would appreciate the input of someone more knowledgeable than myself.
- BorisTheBrave 6y agoPerhaps ±2.4 is the estimated population standard deviation of the tweeted group, rather than a confidence interval. The confidence interval could be the smaller value you seein the charts. Two distributions can overlap a lot and still have a low p-value for anova etc if there are enough data points.
- Will_Do 6y agoYeah I agree this looks like one of those times where the researches aren't interested in doing the stats at all so just mail it in. Just ignoring the stats though the results seem pretty solid. 3x the number of citations with ~100 observations. If the tweets were truly randomly assigned and there aren't big outliers driving results (big ifs, not gonna go deep enough to find out), their conclusion should be fine.