4 ms·

However, so far no efforts have been made to actually correlate the program's judgment with the mental state of the writers. So, it's never been tested? Say w

by rada 16y ago

However, so far no efforts have been made to actually correlate the program's judgment with the mental state of the writers. So, it's never been tested? Say what?

dinedal 16y ago
According to the article, it has been tested, but only to see if psychologists also agree if the text was written by someone that was depressed. No one actually looked at the author and determined if the author was depressed. In otherwords, the program agrees with psychologists 78% of the time if a sample of text may indicate the author is depressed. We're not close at all to a magical diagnosis machine...
raganwald 16y ago
I read the post to mean that the computer program correlated highly with the opinion of a panel of human experts when both are given the writing to study. What hasn't been established is the correlation between the program and the mental state of the authors as measured using more conventional methods.
- rada 16y ago
  Yes, I saw that too, I am a big fan of reading before commenting :) However, I don't think substituting one human error (that of the programmers) by another human error (that of the psychologists) proves anything. You could say that the second group is more qualified as domain experts but it still doesn't make up for complete lack of real life testing. If it did, you wouldn't have to conduct human trials for drugs - you'd just get a bunch of doctors to say that they think the drug works. The fact that the program purports to detect depression, something that amounts to artificial intelligence, makes me especially skeptical. Not to mention the software author's example of using combinations of the word "black" with terms describing "sleep deprivation". By this definition, anyone who blogs about, say, New York Fashion Week, is depressed.
  - samatman 16y ago
    This problem could be mitigated by testing the panel, rather than testing the internet sources. It's a non-trivial problem to get a representative sample of bloggers to submit to a valid screening for depression. On the other hand, collecting a data set of writings by depressed vs. non-depressed people is easier, such data being (I suspect) readily available. If the panel of experts can correlate between depressed writing and depressed people, then there is validity to their correlations of the writing of unknown people, without having to screen the unknowns separately for depression. I suspect even a panel of non-experts could screen writing into depressed and non-depressed. Humans are pretty good at detecting malaise in others.

by rada 16y ago

However, so far no efforts have been made to actually correlate the program's judgment with the mental state of the writers. So, it's never been tested? Say what?

dinedal 16y ago

According to the article, it has been tested, but only to see if psychologists also agree if the text was written by someone that was depressed. No one actually looked at the author and determined if the author was depressed. In otherwords, the program agrees with psychologists 78% of the time if a sample of text may indicate the author is depressed. We're not close at all to a magical diagnosis machine...

raganwald 16y ago

I read the post to mean that the computer program correlated highly with the opinion of a panel of human experts when both are given the writing to study. What hasn't been established is the correlation between the program and the mental state of the authors as measured using more conventional methods.

rada 16y ago
Yes, I saw that too, I am a big fan of reading before commenting :) However, I don't think substituting one human error (that of the programmers) by another human error (that of the psychologists) proves anything. You could say that the second group is more qualified as domain experts but it still doesn't make up for complete lack of real life testing. If it did, you wouldn't have to conduct human trials for drugs - you'd just get a bunch of doctors to say that they think the drug works. The fact that the program purports to detect depression, something that amounts to artificial intelligence, makes me especially skeptical. Not to mention the software author's example of using combinations of the word "black" with terms describing "sleep deprivation". By this definition, anyone who blogs about, say, New York Fashion Week, is depressed.
- samatman 16y ago
  This problem could be mitigated by testing the panel, rather than testing the internet sources. It's a non-trivial problem to get a representative sample of bloggers to submit to a valid screening for depression. On the other hand, collecting a data set of writings by depressed vs. non-depressed people is easier, such data being (I suspect) readily available. If the panel of experts can correlate between depressed writing and depressed people, then there is validity to their correlations of the writing of unknown people, without having to screen the unknowns separately for depression. I suspect even a panel of non-experts could screen writing into depressed and non-depressed. Humans are pretty good at detecting malaise in others.