3 ms·
I disagree with the analysis of this article. In a typical machine learning process, the response variable stays the same (at a distributional level) but you c
by gcmac 9y ago
I disagree with the analysis of this article. In a typical machine learning process, the response variable stays the same (at a distributional level) but you cycle through candidate models. So regardless of whatever the class distributions are, a higher AUC score indicates a better model.
It might be true that the classifier performance is worse on an imbalanced data set (with the same AUC score) than a balanced one but that just reflects the fact that classifiers are harder to build for imbalanced data.
- letitgo12345 9y agoHaving a better AUC score does not guarantee a better AUPR score. So a model with better AUC is not universally "better" See http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.98.4362&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.98....
- eanzenberg 9y agoNo, the point is for an imbalanced set you literally don't care about the model performance where the false-positive-rate is substantial. IE, let's say you have 1% true-hits in your data and run the classifier at an FPR of 5%, that means you are generating ~5 false-positives which is insane to do! That's why most of the ROC curve is useless for imbalanced sets. That's why I prefer precision/recall graphs as does the OP.