3 ms·

I could have acheived better accuracy (87%) by simply calling everyone a non-murderer.

• the_cat_kittles 12y ago
  two people have the opportunity to invest in 20 startups. the first person says no to all of them, and the second person invests in all 20 of them. all but one go bankrupt. the first person proudly proclaims "i can avoid bad investments with 95% probability!"... and the second is a millionaire.
  • reinhardt 12y ago
    Nobody became millionaire by calling everyone a murderer.
    • the_cat_kittles 12y ago
      just pointing out that the "success" of your model depends heavily on how you choose your evaluation criteria. to me, 87% correctly classified seems irrelevant because you can just change the distribution of murderers and non murders and your model would fail. it seems more interesting to me to focus on, say, the rate the model successfully classifies murders.
      • slavik81 12y ago
        The risk structure is different. When you make an investment in a company that turns out not to be successful, you wasted your money. If you persecute someone who would never have murdered anyone, you may have done great and irreversible harm an innocent man. The rate at which it correctly classifies non-murderers is more important for this measure to be useful.
• PeterisP 12y ago
  That's why naive "error rate" is not useful and not used measurement for any classification that's not like 50/50 but more like a "needle in haystack". A more useful measure is F-score (https://en.wikipedia.org/wiki/F1_score https://en.wikipedia.org/wiki/F1_score) according to which calling everyone a non-murderer would give 0%, but the method described in article ("81.29% overall accuracy, 80.00% specificity, and 81.48% sensitivity") would give 81%.