6 ms·
Courtroom AI system claims almost 90 percent accuracy in detecting lies
- ColinWright 9y agoI'm reminded of the scene is "Ex Machina". If you've seen it you'll know what I mean. If you haven't, I won't spoil it for you. But it's (semi-)relevant.
- PhantomGremlin 9y agoTitle is very misleading. These are "pretend" courtrooms. No data from actual legal proceedings. From the article: This was based on evaluations of 104 mock courtroom videos featuring actors instructed to be either deceptive or truthful.
- tj-teej 9y agoAI used for selecting Courtroom Extras for Law and Order SVU Season 31.
- wadkar 9y agoThis is interesting! The ground truth for the study couldn’t have been far removed from the “ground truth”. I am afraid this appears to me as a pattern where any dataset whatsoever is thrown at ML algorithms and the results are heralded as the new way of analyzing data. There is no regard to the source of the data, it’s deficiencies and utter disregard to its applicability to the problem at hand. See the question generation related deep learning papers based on SQUAD dataset.
- alphaalpha101 9y ago90% is not very high.
- gatmne 9y agoalphaalpha101's now dead comment is reasonable: Denying justice in one out of ten cases is unacceptably high.
- grzm 9y agoYou're extrapolating a lot here: this is a demonstration in a mock environment. There's no indication that this is going to be rolled out as-is and used as the final arbiter of justice. Yes, denying justice 10% of the time is unacceptable. But that's not what's being presented here. As an aside, the comment you're referring to was dead on arrival. It looks like that account has been banned.
- Simon_says 9y agoI agree. But what's the error rate now?
- jgamman 9y agoit's the 10% i'm worried about - what's the false positive rate? priors or it didn't happen...
- jgalt212 9y agoWhat's accuracy of lie detector machines?
- justinpombrio 9y agoI don't think they've been shown to be better than placebo.
- harry8 9y agoIf you believe they might work, the operator can do a good cop/bad cop routine with the machine to get you to admit to stuff, frequently stuff that never happened. But they could do the same with a magic rock if you believed in that.
- mikeash 9y agoApparently this has been done with some success by using a photocopier as the “lie detector”: https://newrepublic.com/article/38982/the-wire-ripped-real-life https://newrepublic.com/article/38982/the-wire-ripped-real-l...
- jstarfish 9y agoTerrible (50%?). They're so bad their results haven't been admissible in court for years.
- foolrush 9y agoExactly, and the statistic of 50% doesn't portray the actual reality. The better fact is to cite that a monkey is as accurate as a polygraph machine. Remember that 50% numerical accuracy is _because it cannot be any lower_. If it could, it would be inversely utilized for accuracy.
- 0xcde4c3db 9y agoAccuracy of measuring what? Despite the term, they don't actually detect lies. At best, some of them can detect physiological "events", as when a particular question or word induces acute anxiety. Most interpretation of those events is highly subjective. The closest thing I've heard of to an objective method, the guilty knowledge test, doesn't center on evaluating the truth of the suspect's statements at all. Basically, the authorities know certain confidential facts about the case, such as the location of a wound, characteristics of a weapon, characteristics of an item stolen by the perpetrator, non-public "calling card" of a serial killer, method used to break into a building, etc.. The interviewer (who should be unaware of the true fact, to prevent something like the Clever Hans effect) is given a list including the true fact and plausible alternatives to incorporate into the questions. The idea is that a reaction specifically to the true fact indicates that the suspect knows something that only the actual perpetrator would know. As far as I know, interpretation of these results is still ultimately subjective because there is simply too much variability in human psychology and physiology to define a useful objective standard of when the test is "failed".
- late2part 9y agoAlternate headline: Best Courtroom AI wrongly accuses 1 in 10 people of lying.
- mc32 9y agoBut what is the Jury's alternative (unassisted) effectiveness?
- harry8 9y agoI will be utterly astounded if this "study" is replicated. I will not be remotely surprised if they sell it the gullible fools in public service who don't treat lie detectors with the utter contempt they deserve. What are currently the best tech buzzwords that might convince people you have magic pixie dust for sale?
- umanwizard 9y agoWhat is your skepticism based on?
- StanislavPetrov 9y agoBased on the concept that you should always be skeptical of everything, and even more skeptical when it comes to fantastic claims.
- throwaway613834 9y agoI'm skeptical that it's good to be skeptical of everything...
- StanislavPetrov 9y agoThat's not surprising. Unfortunately you are in the majority.
- drdeca 9y agoIf the idea is true, then wouldn't it actually be good? Or, I guess there may be a distinction between "it is good to, for every thing, to be skeptical of that thing" and "for every thing, it is good to be skeptical of that thing". I consider the statement that "for every statement, it is good to be skeptical of that statement, regardless of what other statements that one is or is not skeptical of" to be doubtful. It seems fairly clear that skepticism which is directed in a specific way can be harmful?
- 9y ago
- vowelless 9y agoWouldn't this violate the Fifth Amendment (self incrimination clause) if actually used?
- gruez 9y agoyou can avoid all this by not testifying.
- wadkar 9y agoInteresting point though I am not sure how the 5th amendment would apply here. Specifically, I thought that 5th amendment allows you to not answer a question or not testify. After you say something, it looks like using it to make a case against you is a fair game.
- foolrush 9y agoNot designed by sociologists. More rubbish out of the polygraph playbook. AI is the modern Mechanical Turk when applied to the complex nonlinearities of human behavior.
- deepnotderp 9y agoI tried for a while to think of a good retort to this article, I couldn't. So instead: This is fucking retarded. They should institute a law that if you are using machine learning to make major decisions about people's lives then you need to take a test about basic ML (test sets, validation sets, etc.)
- wadkar 9y agoPerhaps one could insist for an explanation and reasonable defense of the outcome by these ML algorithms. Would that be a feasible retort?
- deleted 9y ago[deleted]
- pg_bot 9y agoEven if they can, anyone who understands Bayes' theorem will tell you that the false positive rate would be way too high for this to be used in real life.
- yorwba 9y agoI would have liked to read the actual paper, but the arXiv link leads to something completely unrelated: https://arxiv.org/abs/1712.05526 https://arxiv.org/abs/1712.05526 Does anyone have a correct link?
- advisedwang 9y agoThey meant https://arxiv.org/abs/1712.04415 https://arxiv.org/abs/1712.04415 (based on a full text arxiv search of the text mentioned in article)
- AstralStorm 9y agoThe project is known as DARE. Interesting classifier but not really that great. Try it out, it is open source so far. Just like many proof of concept it starts to rapidly fail when trained on too much data and otherwise generalize only so so. This is essentially some feature engineering thrown at an SVM / GNN. What makes the results even tougher to reproduce is that the training set is not provided, only resulting matrices. It is telling when different classifiers work best for supervised and auto cases...
- chrisco255 9y agoWhat about half truths?
- kapauldo 9y agoIt's data science, its got to be true.