4 ms·

Youre right that getting rates of mental disorders at the population level is a useful application. Checkout for example the hedonometer: https://hedonometer.or

Youre right that getting rates of mental disorders at the population level is a useful application. Checkout for example the hedonometer: https://hedonometer.org/timeseries/en_all/?from=2021-12-05&to=2022-03-04 https://hedonometer.org/timeseries/en_all/?from=2021-12-05&t.... People have also done population-level research showing that bad weather affect various seniment-based metrics of mental health: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0195750 https://journals.plos.org/plosone/article?id=10.1371/journal...

apwheele 5y ago
This doesn't work as a reasonable objective measure of population mental health disorders for several reasons. 1) The oversampling of the training set makes it uncalibrated when applied to the general population (to recalibrate you need an estimate of the prevalence to begin with, which sort of defeats the purpose). 2) Online posts are not a random sample of the population. (Perhaps this is solveable with some poststratification of the estimates, although requires demographic data on the poster.) If you take self reports that the researchers used to define disorders at face value, those would make more sense than using this model. These text based models are so superficial, when applied to mass datasets with low prevalence of the underlying condition, they will ultimately result in very low positive predictive values (e.g. flag 100 people, if the model is good will only get 5/100 as actual mental health problems). As version_five asks, it is hard to imagine any reasonable use of the model given such low positive predictive values (which imply incredibly high false positive rates).
mistrial9 5y ago
> getting rates of mental disorders at the population level is a useful application this post sounds dangerous to me -- there are vast differences in a legal judgement and a medical judgement; thirdly, political policy levers are sometimes used in ways that make very little sense, due to situational factors. It is no secret that humans are competitive, tend towards violence, and political oppression exists to some extent, of some groups, almost everywhere. A "medical judgement" used for legal or worse, political purposes, I claim, is dangerous. The post above this one appears to make no distinctions and leans towards "yes, do this."

apwheele 5y ago

This doesn't work as a reasonable objective measure of population mental health disorders for several reasons. 1) The oversampling of the training set makes it uncalibrated when applied to the general population (to recalibrate you need an estimate of the prevalence to begin with, which sort of defeats the purpose). 2) Online posts are not a random sample of the population. (Perhaps this is solveable with some poststratification of the estimates, although requires demographic data on the poster.) If you take self reports that the researchers used to define disorders at face value, those would make more sense than using this model. These text based models are so superficial, when applied to mass datasets with low prevalence of the underlying condition, they will ultimately result in very low positive predictive values (e.g. flag 100 people, if the model is good will only get 5/100 as actual mental health problems). As version_five asks, it is hard to imagine any reasonable use of the model given such low positive predictive values (which imply incredibly high false positive rates).

mistrial9 5y ago

> getting rates of mental disorders at the population level is a useful application this post sounds dangerous to me -- there are vast differences in a legal judgement and a medical judgement; thirdly, political policy levers are sometimes used in ways that make very little sense, due to situational factors. It is no secret that humans are competitive, tend towards violence, and political oppression exists to some extent, of some groups, almost everywhere. A "medical judgement" used for legal or worse, political purposes, I claim, is dangerous. The post above this one appears to make no distinctions and leans towards "yes, do this."