6 ms·
A single accuracy rate is not that useful when looking at ML problems. There is always a tradeoff between false positives and false negatives depending on the t
by zhobbs 8y ago
A single accuracy rate is not that useful when looking at ML problems. There is always a tradeoff between false positives and false negatives depending on the threshold selected. Amazon is guilty of this in this article, saying 0% false positive rate when you set the threshold to 99%. However, it doesn't say how many faces it successfully matched, it's easy to have a 0% false positive rate if you just say "no match" for every photo.
The best practice is to show a ROC curve, which shows the tradeoffs of selecting a given threshold: https://en.wikipedia.org/wiki/Receiver_operating_characteristic https://en.wikipedia.org/wiki/Receiver_operating_characteris...
- smittywerben 8y agoAfter Pearl Harbor, some wanted to get rid of early-warning radar systems. The military created the ROC to screen for operators who might misidentify enemy planes. The radar did fine, the interpretation was wrong.
- joshgel 8y agoInteresting, do you have a source? I'm interested in misjudgment and cognitive errors, so i'd like to read about this.
- smittywerben 8y agoIt's mostly Wikipedia - the history is a bit scattered. Probably better sources out there. [0] https://en.wikipedia.org/wiki/Receiver_operating_characteristic#History https://en.wikipedia.org/wiki/Receiver_operating_characteris... [1] https://en.wikipedia.org/wiki/SCR-270 https://en.wikipedia.org/wiki/SCR-270 [2] https://archive.org/stream/Vol1PlansAndEarlyOperations#page/n335/ https://archive.org/stream/Vol1PlansAndEarlyOperations#page/...
- sgt101 8y agoAssuming that the data set that test and training are drawn from is representative of the data that you see in production. Which it never is.