4 ms·

I am not comparing it to a human error. There is a conventional wisdom that humans make mistakes, and we have redundancy built into various systems to counter t

I am not comparing it to a human error. There is a conventional wisdom that humans make mistakes, and we have redundancy built into various systems to counter the same. With machines and tech, the expectation is that they are accurate (which is not true, but mostly true in the important ways). The reason I brought it up is to hope we have that redundancy in this case and future implementations of AI as well. (Horizon had a bug in calculation system - which seems easy and straightforward and hence unexpected and undetected.) A typical example would be breath analysers. If humans were to judge who is drunk, there is a high error rate. With tech, the error rate is low, but if there is a bug in one of the equipments, many wrong and unnecessary arrests might be made.

acdha 2y ago
> With machines and tech, the expectation is that they are accurate (which is not true, but mostly true in the important ways). I find this an interesting comment with your choice of examples as there is an extensive legal history about false positives (medical conditions, mouthwash, etc.) and the need to show maintenance and calibration records. I was on a jury back in the 2000s where the San Diego police had to have someone from the crime lab testify about their process for managing their breathalyzers, and we ended up unanimously finding the guy not guilty on that charge because they’d messed up the process and had been a year late on calibration testing despite everyone agreeing he was drunk (based on his own testimony, receipts, and the bar staff, he’d had 3 double vodkas in the hour before getting behind the wheel, so we found him guilty on the other prong of California’s DUI statute - which is exactly what his attorney told us afterwards had been his prediction).
- ankit219 2y ago
  Your experience is very helpful. I took a theoretical example since where I live breath analysers are how you get fined for drunk driving. I didn't think it was possible to contest that. I would rather that these things happen preemptively so that there are no such challenges to fight or tickets to contest just because an AI flagged something and humans went ahead with it. I mean that other tech is deterministic, while AI is probabilistic, but people still havent updated their models for such tech. And we need to keep those examples in mind.

acdha 2y ago

> With machines and tech, the expectation is that they are accurate (which is not true, but mostly true in the important ways). I find this an interesting comment with your choice of examples as there is an extensive legal history about false positives (medical conditions, mouthwash, etc.) and the need to show maintenance and calibration records. I was on a jury back in the 2000s where the San Diego police had to have someone from the crime lab testify about their process for managing their breathalyzers, and we ended up unanimously finding the guy not guilty on that charge because they’d messed up the process and had been a year late on calibration testing despite everyone agreeing he was drunk (based on his own testimony, receipts, and the bar staff, he’d had 3 double vodkas in the hour before getting behind the wheel, so we found him guilty on the other prong of California’s DUI statute - which is exactly what his attorney told us afterwards had been his prediction).

ankit219 2y ago
Your experience is very helpful. I took a theoretical example since where I live breath analysers are how you get fined for drunk driving. I didn't think it was possible to contest that. I would rather that these things happen preemptively so that there are no such challenges to fight or tickets to contest just because an AI flagged something and humans went ahead with it. I mean that other tech is deterministic, while AI is probabilistic, but people still havent updated their models for such tech. And we need to keep those examples in mind.