4 ms·
I believe they tackle this exact bias. From the Sampling Effects section: > Second, the choice of CDS n-grams could lead to a "recency bias” in our results, ex
by pedrosbmartins 4y ago
I believe they tackle this exact bias. From the Sampling Effects section:
> Second, the choice of CDS n-grams could lead to a "recency bias” in our results, explaining their rise in prevalence in recent decades. We control for this effect with a null model that samples random n-grams more frequently from recent books, due to rapidly increasing publication volume since 1895, thereby inducing a bias toward more recent language. We observe increases of CDS n-gram prevalence well above levels predicted by this null model
- bryanrasmussen 4y ago> thereby inducing a bias toward more recent language. We observe increases of CDS n-gram prevalence well above levels predicted by this null model I don't get why this would work? I get the null model predicting a bias X, and I guess they have a greater bias X + Y, but I don't see how that handles their choice of cognitive distortion signifiers being biased? I mean isn't it likely their choice of signifiers matches to Y?
- lumost 4y agothey are claiming that their basket of distortion N-Grams became more common faster than other randomly sampled n-grams from recent works. This seems like an interesting approach to controlling for the bias, but I'd expect the random sampling would bias lower than a specific sample as a random sampling of n-grams would pick up a lot of english grammar which hasn't changed in many years.
- geysersam 4y agoMaybe they should construct a list of n-grams indicative of "non-depressed"/healthy people, and run the same analysis on that list.
- layer8 4y agoThey also address this to some degree in the Language effects section.