4 ms·
> Understanding confidence thresholds is essential to using software like this. Thinking they don't "utilize" one means they don't understand the first thing ab
by yorwba 8y ago
> Understanding confidence thresholds is essential to using software like this. Thinking they don't "utilize" one means they don't understand the first thing about it.
Thresholding is not the only way to make use of a classifier that ouputs a confidence score. In particular, the slide shown in the article indicates that they use the confidence score to sort search results in descending order. That means that there is no fixed threshold and the PIO's statement is not worrisome at all.
The system described is much less dangerous (in terms of overconfident decisions) than one that only returns results matching a threshold, because it means that the user also occasionally sees low-confidence results and learns that they can't simply defer decisions to the machine.
- b_tterc_p 8y agoUnless the cops aren’t exactly virtuous actors... which they frequently are not. You can’t assume they care if the results are good. It is another tool to justify police action with a low true positive rate.
- pdkl95 8y ago> It is another tool to justify police action "We were following the AI leads" is the perfect excuse for parallel construction[1] (evidence laundering). If Rekognition supports limiting the search to the people in a specific town (or other small group), it might be possible to force someone to be somewhere on the list of matches. That would give bad actors the perfect tool for manufacturing "probable cause". [1] https://en.wikipedia.org/wiki/Parallel_construction https://en.wikipedia.org/wiki/Parallel_construction
- Spivak 8y agoIf parallel construction is in your threat model then does the existence of a slightly better face search database really matter?
- craftinator 8y agoDamn, beat me to it
- muttled 8y agoI've seen how capricious some can be with their work tools. I dated a girl whose dad ran a background check on everyone she dated without their knowledge or consent. I was 19 at the time and it seemed wrong, but reporting one of the higher up police officers in my town who now knew all my details seemed like a freedom limiting move.
- philipodonnell 8y agoStill worrisome. You're assuming that there is a wide distribution of confidence levels in every search. Its very possible with grainy footage that no result is above a reasonable threshold, but the results will still show an ordered list. Without _some_ threshold or training in what these kind of scores mean, its easy for a user to think "well they were the best match" and take a negative action like bringing them in for questioning or introducing the match as evidence, even if its only 10% +/- 5%.
- Ntrails 8y agoSeeing a single result just above threshold leads a person to believe it's a match. Seeing the 200 people within a few points of the threshold might lead someone to think it was actually completely indeterminate with a large number of potential suspects not far from each other. (I could be missing something here)