3 ms·

how does it compare versus human error?

lolinder 2y ago
What the exact error rate is isn't relevant to OP's concern, which is that operators of the tooling will treat it as having a 0% error rate in a way that they don't with human judgment. We've already seen this play out with algorithms designed to predict recidivism or what have you—law enforcement and judicial officials place undue weight on the outputs of algorithms and trust them implicitly where they might ask questions of a human expert.
- acdha 2y ago
  > which is that operators of the tooling will treat it as having a 0% error rate in a way that they don't with human judgment This is possible but we have decades of camera enforcement experience suggesting that courts are aware that these systems are not as reliable as the manufacturers claim. People have been challenging photo tickets since the previous century and there’s a well-established process for challenging them.
antegamisou 2y ago
Techbros always love citing the supposedly devastating human fallibility yet experience shows poorly configured/developed automated systems are much more of a concern.
- pompino 2y ago
  Nice straw-man, but getting back to reality, nobody in this thread has put forth the belief that automated systems are infallible, or is arguing that point. Also the fact that humans are fallible, hasn't stopped us from setting up judicial systems based on human judgement - which work pretty well in several scenarios.
  - antegamisou 2y ago
    sTrAwmAn
ankit219 2y ago
I am not comparing it to a human error. There is a conventional wisdom that humans make mistakes, and we have redundancy built into various systems to counter the same. With machines and tech, the expectation is that they are accurate (which is not true, but mostly true in the important ways). The reason I brought it up is to hope we have that redundancy in this case and future implementations of AI as well. (Horizon had a bug in calculation system - which seems easy and straightforward and hence unexpected and undetected.) A typical example would be breath analysers. If humans were to judge who is drunk, there is a high error rate. With tech, the error rate is low, but if there is a bug in one of the equipments, many wrong and unnecessary arrests might be made.
- acdha 2y ago
  > With machines and tech, the expectation is that they are accurate (which is not true, but mostly true in the important ways). I find this an interesting comment with your choice of examples as there is an extensive legal history about false positives (medical conditions, mouthwash, etc.) and the need to show maintenance and calibration records. I was on a jury back in the 2000s where the San Diego police had to have someone from the crime lab testify about their process for managing their breathalyzers, and we ended up unanimously finding the guy not guilty on that charge because they’d messed up the process and had been a year late on calibration testing despite everyone agreeing he was drunk (based on his own testimony, receipts, and the bar staff, he’d had 3 double vodkas in the hour before getting behind the wheel, so we found him guilty on the other prong of California’s DUI statute - which is exactly what his attorney told us afterwards had been his prediction).
  - ankit219 2y ago
    Your experience is very helpful. I took a theoretical example since where I live breath analysers are how you get fined for drunk driving. I didn't think it was possible to contest that. I would rather that these things happen preemptively so that there are no such challenges to fight or tickets to contest just because an AI flagged something and humans went ahead with it. I mean that other tech is deterministic, while AI is probabilistic, but people still havent updated their models for such tech. And we need to keep those examples in mind.

What the exact error rate is isn't relevant to OP's concern, which is that operators of the tooling will treat it as having a 0% error rate in a way that they don't with human judgment. We've already seen this play out with algorithms designed to predict recidivism or what have you—law enforcement and judicial officials place undue weight on the outputs of algorithms and trust them implicitly where they might ask questions of a human expert.

acdha 2y ago
> which is that operators of the tooling will treat it as having a 0% error rate in a way that they don't with human judgment This is possible but we have decades of camera enforcement experience suggesting that courts are aware that these systems are not as reliable as the manufacturers claim. People have been challenging photo tickets since the previous century and there’s a well-established process for challenging them.

antegamisou 2y ago

Techbros always love citing the supposedly devastating human fallibility yet experience shows poorly configured/developed automated systems are much more of a concern.

pompino 2y ago
Nice straw-man, but getting back to reality, nobody in this thread has put forth the belief that automated systems are infallible, or is arguing that point. Also the fact that humans are fallible, hasn't stopped us from setting up judicial systems based on human judgement - which work pretty well in several scenarios.
- antegamisou 2y ago
  sTrAwmAn

ankit219 2y ago

I am not comparing it to a human error. There is a conventional wisdom that humans make mistakes, and we have redundancy built into various systems to counter the same. With machines and tech, the expectation is that they are accurate (which is not true, but mostly true in the important ways). The reason I brought it up is to hope we have that redundancy in this case and future implementations of AI as well. (Horizon had a bug in calculation system - which seems easy and straightforward and hence unexpected and undetected.) A typical example would be breath analysers. If humans were to judge who is drunk, there is a high error rate. With tech, the error rate is low, but if there is a bug in one of the equipments, many wrong and unnecessary arrests might be made.

acdha 2y ago
> With machines and tech, the expectation is that they are accurate (which is not true, but mostly true in the important ways). I find this an interesting comment with your choice of examples as there is an extensive legal history about false positives (medical conditions, mouthwash, etc.) and the need to show maintenance and calibration records. I was on a jury back in the 2000s where the San Diego police had to have someone from the crime lab testify about their process for managing their breathalyzers, and we ended up unanimously finding the guy not guilty on that charge because they’d messed up the process and had been a year late on calibration testing despite everyone agreeing he was drunk (based on his own testimony, receipts, and the bar staff, he’d had 3 double vodkas in the hour before getting behind the wheel, so we found him guilty on the other prong of California’s DUI statute - which is exactly what his attorney told us afterwards had been his prediction).
- ankit219 2y ago
  Your experience is very helpful. I took a theoretical example since where I live breath analysers are how you get fined for drunk driving. I didn't think it was possible to contest that. I would rather that these things happen preemptively so that there are no such challenges to fight or tickets to contest just because an AI flagged something and humans went ahead with it. I mean that other tech is deterministic, while AI is probabilistic, but people still havent updated their models for such tech. And we need to keep those examples in mind.