3 ms·

I read the post to mean that the computer program correlated highly with the opinion of a panel of human experts when both are given the writing to study. What

I read the post to mean that the computer program correlated highly with the opinion of a panel of human experts when both are given the writing to study. What hasn't been established is the correlation between the program and the mental state of the authors as measured using more conventional methods.

rada 16y ago
Yes, I saw that too, I am a big fan of reading before commenting :) However, I don't think substituting one human error (that of the programmers) by another human error (that of the psychologists) proves anything. You could say that the second group is more qualified as domain experts but it still doesn't make up for complete lack of real life testing. If it did, you wouldn't have to conduct human trials for drugs - you'd just get a bunch of doctors to say that they think the drug works. The fact that the program purports to detect depression, something that amounts to artificial intelligence, makes me especially skeptical. Not to mention the software author's example of using combinations of the word "black" with terms describing "sleep deprivation". By this definition, anyone who blogs about, say, New York Fashion Week, is depressed.
- samatman 16y ago
  This problem could be mitigated by testing the panel, rather than testing the internet sources. It's a non-trivial problem to get a representative sample of bloggers to submit to a valid screening for depression. On the other hand, collecting a data set of writings by depressed vs. non-depressed people is easier, such data being (I suspect) readily available. If the panel of experts can correlate between depressed writing and depressed people, then there is validity to their correlations of the writing of unknown people, without having to screen the unknowns separately for depression. I suspect even a panel of non-experts could screen writing into depressed and non-depressed. Humans are pretty good at detecting malaise in others.

rada 16y ago

Yes, I saw that too, I am a big fan of reading before commenting :) However, I don't think substituting one human error (that of the programmers) by another human error (that of the psychologists) proves anything. You could say that the second group is more qualified as domain experts but it still doesn't make up for complete lack of real life testing. If it did, you wouldn't have to conduct human trials for drugs - you'd just get a bunch of doctors to say that they think the drug works. The fact that the program purports to detect depression, something that amounts to artificial intelligence, makes me especially skeptical. Not to mention the software author's example of using combinations of the word "black" with terms describing "sleep deprivation". By this definition, anyone who blogs about, say, New York Fashion Week, is depressed.

samatman 16y ago
This problem could be mitigated by testing the panel, rather than testing the internet sources. It's a non-trivial problem to get a representative sample of bloggers to submit to a valid screening for depression. On the other hand, collecting a data set of writings by depressed vs. non-depressed people is easier, such data being (I suspect) readily available. If the panel of experts can correlate between depressed writing and depressed people, then there is validity to their correlations of the writing of unknown people, without having to screen the unknowns separately for depression. I suspect even a panel of non-experts could screen writing into depressed and non-depressed. Humans are pretty good at detecting malaise in others.