4 ms·
The results are actually really really bad. From the article: > Nevertheless, less than 4% of new songs will become hits So 96% of songs are not-hits. If the
by KMnO4 3y ago
The results are actually really really bad.
From the article:
> Nevertheless, less than 4% of new songs will become hits
So 96% of songs are not-hits. If the model predicts that no song will be a hit, then its accuracy is already 96%.
- screamingninja 3y agoWouldn't the accuracy be zero in that case, since it would identify none of the hit songs correctly? I feel that identify hit songs is harder than identifying non-hit songs.
- KMnO4 3y agoTP = correct hit songs = 0 TN = correct non-hits = 96 FP = incorrect hits = 4 FN = incorrect non-hits = 0 Accuracy = (0+96)/(0+96+4+0) = 96/100 This is just a problem with using the metric of accuracy. If your classes are unbalanced, then a few “errors” aren’t going to affect your accuracy.
- Spivak 3y agoWhich is why you're not interested in the hit/not hit accuracy. You're interested in "if model says it's a hit how often is it right?" Imagine a Monty Hall situation with 100 doors, you can predict with 99% accuracy which doors have goats but you only have 1% accuracy on cars.
- Aachen 3y agoRelevant https://web.archive.org/web/20210112213001/https://dilbert.com/strip/2001-10-25 https://web.archive.org/web/20210112213001/https://dilbert.c...
- seanw265 3y agoAs other commenters have pointed out, accuracy is not a great metric to use here. A confusion matrix [1] is needed to fully capture the results. I'm surprised that one was not prominently displayed in this article. They are standard practice for evaluating binary classifiers. [1] https://www.dataschool.io/simple-guide-to-confusion-matrix-terminology/ https://www.dataschool.io/simple-guide-to-confusion-matrix-t...