4 ms·

The sad thing is that the team responsible for this bit of machine learning are probably very proud of their 99% accurate algorithm. It probably is 99% accur

by nickcw 6y ago

The sad thing is that the team responsible for this bit of machine learning are probably very proud of their 99% accurate algorithm. It probably *is* 99% accurate - there is a lot of fraudulent activity online. That 1% false positive rate really hurts though, destroying people's livelihoods. I think that people developing machine learning models sometimes don't think about the consequences of the false positives enough. There is a great section in David Spiegelhalter's book The Art of Statistics which got me thinking about this.

Edd314159 6y ago
I agree with you in this instance, you make a good point. 99% accuracy sounds great on paper, but that's a very painful 1% remainder - and in this case, there's no way a rational human would have acted in the same disastrous way the AI did. That said, in many other applications of AI, a 1% error rate is probably a huge success. AI does not need to be perfect to be considered successful, it only needs to do better than a human. In this example, AI failed miserably at that, but there are plenty of applications where AI errs far less frequently than humans do at the same task.
klmadfejno 6y ago
To be pedantic def is_fraud(x): return False is in all likelihood a greater than 99% accurate algo.
TomSwirly 6y ago
Having even a quite high false positive rate would be totally acceptable, if the process involved humans on the Large Faceless Company side. You'd get an email, "Hello, there might be a problem, here are the details" and if you made some good faith attempt to reply, be able to talk to an actual person, who could potentially escalate your issue.
prepend 6y ago
The problem here is you need two algorithms. Their current algorithm might be 99% sensitive to detecting fraud. But they need a 99.99999999% algorithm that’s specific to not fraud. So by running then positives through the second one, they can detect the false positives and lessen these episodes.

The sad thing is that the team responsible for this bit of machine learning are probably very proud of their 99% accurate algorithm. It probably *is* 99% accur

by nickcw 6y ago

Edd314159 6y ago

I agree with you in this instance, you make a good point. 99% accuracy sounds great on paper, but that's a very painful 1% remainder - and in this case, there's no way a rational human would have acted in the same disastrous way the AI did. That said, in many other applications of AI, a 1% error rate is probably a huge success. AI does not need to be perfect to be considered successful, it only needs to do better than a human. In this example, AI failed miserably at that, but there are plenty of applications where AI errs far less frequently than humans do at the same task.

klmadfejno 6y ago

To be pedantic def is_fraud(x): return False is in all likelihood a greater than 99% accurate algo.

TomSwirly 6y ago

Having even a quite high false positive rate would be totally acceptable, if the process involved humans on the Large Faceless Company side. You'd get an email, "Hello, there might be a problem, here are the details" and if you made some good faith attempt to reply, be able to talk to an actual person, who could potentially escalate your issue.

prepend 6y ago

The problem here is you need two algorithms. Their current algorithm might be 99% sensitive to detecting fraud. But they need a 99.99999999% algorithm that’s specific to not fraud. So by running then positives through the second one, they can detect the false positives and lessen these episodes.