5 ms·

> but Amazon says it recommends at least a 95 percent threshold for law enforcement applications where a false ID might have more significant consequences. Tha

> but Amazon says it recommends at least a 95 percent threshold for law enforcement applications where a false ID might have more significant consequences. That's not in https://aws.amazon.com/rekognition/faqs/ https://aws.amazon.com/rekognition/faqs/ - while that threshold may minimize false positives, I'm curious if it was in any documentation that the ACLU saw. Or, to the bigger point, is it in any public documentation?

dfundako 8y ago
I would be shocked if LE started using this tech, imprisoned a bunch of people, and used the excuse "Well, we followed the FAQS." I would imagine they would work very closely with Amazon and run a ton of calibration. For the ACLU to say that we used the default settings and it doesn't work correctly is disingenuous.
- EliRivers 8y ago
  I would imagine they would work very closely with Amazon and run a ton of calibration. Do you have any evidence for this? I would have imagined the other way; that it would be badly deployed by ill-trained gung-ho operators who will routinely end up clubbing some innocent guy round the head (if they don't just shoot him) because the magic machine said he was a dangerous killer on the loose. I don't have any evidence my way, but what I imagined is the polar opposite of what you imagined.
- Jweb_Guru 8y ago
  Assuming that law enforcement is going to spend any amount of time being careful about enforcing correct use of forensic tools is hard to reconcile with the history of the American judicial system. For example, there is essentially no scientific evidence that a polygraph test measures anything at all, but polygraph evidence is considered admissible in many courts and used to be admitted in many more. Moreover, rigorous studies of fingerprint evidence have found them to have false positive rates as high as 1 in 18, but they are still often treated as close to infallible in court. I am not sure why you think facial recognition is going to be different.
- delecti 8y ago
  I would not be shocked at all if some LE somewhere used the default settings. It seems absurd to assume otherwise. It's not as though LE has a great track record. https://en.wikipedia.org/wiki/No_Fly_List#False_positives https://en.wikipedia.org/wiki/No_Fly_List#False_positives
  - ocfx 8y ago
    LE won't be given "settings"
- philipodonnell 8y ago
  You are going to be very very shocked my friend!
- spamizbad 8y ago
  LE doesn't have the best track record following best practices. For example, they still rely heavily on notoriously unreliable eye-witness accounts and polygraph tests, two things science has largely debunked. They were slow to let go of phrenology as well.
notyourwork 8y ago
Threshold is a configuration property no different than a dynamo read capacity. Misconfigured properties don’t work the way you expect. I don’t see how documentation can be expected to give black and white guidance to what this threshold should be set to without having more understanding of your use case.
- dragonwriter 8y ago
  > I don’t see how documentation can be expected to give black and white guidance to what this threshold should be set to without having more understanding of your use case. It can say exactly what the Amazon response to the Verge dismissing the ACLU test should, and, in fact, it should say that if Amazon really does have such critical, specific domain-specific recommended settings, rather than it being something made up off the cuff to mitigate bad press.
  - Kalium 8y ago
    You're unquestionably correct. Amazon should spell out in clear, simple, easily accessible language what level of certainty is appropriate for what type of application, giving concrete examples at every point. With that said, it's possible that no amount of clear guidance would have mattered here. The ACLU isn't exactly asking for better docs.
- Kalium 8y ago
  If you're someone with minimal understanding of the use and deployment of technologies, who knows only of this technology as something used to detect faces, then it's a glaring oversight. If you're a researcher making a deliberate choice to use the defaults to simulate a random police officer with no real understanding of their tools, then say as much. And compare with more cautious results.

dfundako 8y ago

I would be shocked if LE started using this tech, imprisoned a bunch of people, and used the excuse "Well, we followed the FAQS." I would imagine they would work very closely with Amazon and run a ton of calibration. For the ACLU to say that we used the default settings and it doesn't work correctly is disingenuous.

EliRivers 8y ago
I would imagine they would work very closely with Amazon and run a ton of calibration. Do you have any evidence for this? I would have imagined the other way; that it would be badly deployed by ill-trained gung-ho operators who will routinely end up clubbing some innocent guy round the head (if they don't just shoot him) because the magic machine said he was a dangerous killer on the loose. I don't have any evidence my way, but what I imagined is the polar opposite of what you imagined.
Jweb_Guru 8y ago
Assuming that law enforcement is going to spend any amount of time being careful about enforcing correct use of forensic tools is hard to reconcile with the history of the American judicial system. For example, there is essentially no scientific evidence that a polygraph test measures anything at all, but polygraph evidence is considered admissible in many courts and used to be admitted in many more. Moreover, rigorous studies of fingerprint evidence have found them to have false positive rates as high as 1 in 18, but they are still often treated as close to infallible in court. I am not sure why you think facial recognition is going to be different.
delecti 8y ago
I would not be shocked at all if some LE somewhere used the default settings. It seems absurd to assume otherwise. It's not as though LE has a great track record. https://en.wikipedia.org/wiki/No_Fly_List#False_positives https://en.wikipedia.org/wiki/No_Fly_List#False_positives
- ocfx 8y ago
  LE won't be given "settings"
philipodonnell 8y ago
You are going to be very very shocked my friend!
spamizbad 8y ago
LE doesn't have the best track record following best practices. For example, they still rely heavily on notoriously unreliable eye-witness accounts and polygraph tests, two things science has largely debunked. They were slow to let go of phrenology as well.

notyourwork 8y ago

Threshold is a configuration property no different than a dynamo read capacity. Misconfigured properties don’t work the way you expect. I don’t see how documentation can be expected to give black and white guidance to what this threshold should be set to without having more understanding of your use case.

dragonwriter 8y ago
> I don’t see how documentation can be expected to give black and white guidance to what this threshold should be set to without having more understanding of your use case. It can say exactly what the Amazon response to the Verge dismissing the ACLU test should, and, in fact, it should say that if Amazon really does have such critical, specific domain-specific recommended settings, rather than it being something made up off the cuff to mitigate bad press.
- Kalium 8y ago
  You're unquestionably correct. Amazon should spell out in clear, simple, easily accessible language what level of certainty is appropriate for what type of application, giving concrete examples at every point. With that said, it's possible that no amount of clear guidance would have mattered here. The ACLU isn't exactly asking for better docs.
Kalium 8y ago
If you're someone with minimal understanding of the use and deployment of technologies, who knows only of this technology as something used to detect faces, then it's a glaring oversight. If you're a researcher making a deliberate choice to use the defaults to simulate a random police officer with no real understanding of their tools, then say as much. And compare with more cautious results.