27 ms·

That 85% 'accuracy' is ambiguous. I couldn't get into the article, but if that number includes false positives, that's not really practical. Suicide is really

by rocqua 10y ago

That 85% 'accuracy' is ambiguous. I couldn't get into the article, but if that number includes false positives, that's not really practical. Suicide is really rare, so it could just be that it picks a 15% of the population that includes al suicidal people. That means the vast majority of those positives are false positives. Basically, its the precision and recall that matters.

rawnlq 10y ago
I think machine learning people use a combination (specifically the harmonic mean) of both when evaluating: https://en.wikipedia.org/wiki/F1_score https://en.wikipedia.org/wiki/F1_score
- dx034 10y ago
  They should use it, but articles often drop that (or similar) numbers. It's easy to get headlines if you report 85% accuracy as it sounds good and is easy to understand. Reporting the proper numbers is harder to understand for people who are not working in the field and you'd quickly discover that the model doesn't actually work.
denzil_correa 10y ago
I'm not even sure why people report ONLY accuracy as an evaluation metric. It means nothing - appropriately weighted F scores, precision, recall give more insight into the system.
- dasboth 10y ago
  Completely agree. Although if your binary classes are perfectly balanced then accuracy is a valid metric, but that's rarely the case. I guess the problem is that an ROC curve wouldn't create flashy headlines.
  - denzil_correa 10y ago
    > I guess the problem is that an ROC curve wouldn't create flashy headlines. You can, you just need to put in a little bit of work. We don't need to specify the exact accuracy in the title.
nommm-nommm 10y ago
I wouldn't call suicide really rare, it's the 10th most common cause of death in the US http://www.cdc.gov/nchs/fastats/leading-causes-of-death.htm http://www.cdc.gov/nchs/fastats/leading-causes-of-death.htm
- hga 10y ago
  Although nowadays you have to view these statistics carefully, for X, Y, and Z causes of death inevitably rise as I, J, and K get knocked down by modern medicine, all we've done to reduce many types of accidental death, from 5 gallon buckets to cars, etc. But, yeah, in this case I agree it's a big problem in the US.

by rocqua 10y ago

rawnlq 10y ago

I think machine learning people use a combination (specifically the harmonic mean) of both when evaluating: https://en.wikipedia.org/wiki/F1_score https://en.wikipedia.org/wiki/F1_score

dx034 10y ago
They should use it, but articles often drop that (or similar) numbers. It's easy to get headlines if you report 85% accuracy as it sounds good and is easy to understand. Reporting the proper numbers is harder to understand for people who are not working in the field and you'd quickly discover that the model doesn't actually work.

denzil_correa 10y ago

I'm not even sure why people report ONLY accuracy as an evaluation metric. It means nothing - appropriately weighted F scores, precision, recall give more insight into the system.

dasboth 10y ago
Completely agree. Although if your binary classes are perfectly balanced then accuracy is a valid metric, but that's rarely the case. I guess the problem is that an ROC curve wouldn't create flashy headlines.
- denzil_correa 10y ago
  > I guess the problem is that an ROC curve wouldn't create flashy headlines. You can, you just need to put in a little bit of work. We don't need to specify the exact accuracy in the title.

nommm-nommm 10y ago

I wouldn't call suicide really rare, it's the 10th most common cause of death in the US http://www.cdc.gov/nchs/fastats/leading-causes-of-death.htm http://www.cdc.gov/nchs/fastats/leading-causes-of-death.htm

hga 10y ago
Although nowadays you have to view these statistics carefully, for X, Y, and Z causes of death inevitably rise as I, J, and K get knocked down by modern medicine, all we've done to reduce many types of accidental death, from 5 gallon buckets to cars, etc. But, yeah, in this case I agree it's a big problem in the US.