4 ms·
My main point is not about aggregate accuracy, it's about comparing confidence across classes of datasets. Edit: the entire point of bayesian approach is that
by paraschopra 8y ago
My main point is not about aggregate accuracy, it's about comparing confidence across classes of datasets.
Edit: the entire point of bayesian approach is that you can make decisions on a loss function where you can make a tradeoff on the (business) cost of making a wrong decision and (business) regret of not making a decision. For real world applications, accuracy is a useless metric. What matters is the net (business) benefit or loss.
- chrisfosterelli 8y agoYour article bolds the following in the introductory paragraph: > My final classifier had accuracy of ~97% on MNIST and it refused to classify white noise and the majority of unrelated (non-MNIST) images. I'm saying that you are making a misleading claim when you say this. It is categorically untrue that you achieve 97% on MNIST with this approach. Accuracy on a standardized test set makes no sense when you select a subset from the test set for maximal accuracy. I'm not saying that your approach doesn't have value or make sense in some business cases, because I think the overall idea is interesting but these statements are not correct.
- pacala 8y agoCorrect. That's why people use https://en.wikipedia.org/wiki/F1_score https://en.wikipedia.org/wiki/F1_score instead of just accuracy.