15 ms·
81% of 'suspects' flagged by Met's police facial recognition technology innocent
- eesmith 7y agoThe Met prefers to measure accuracy by comparing successful and unsuccessful matches with the total number of faces processed by the facial recognition system. According to this metric, the error rate was just 0.1%. Turn the machines off and the error rate will be 0%.
- Traster 7y agoThe Met prefers a metric where adding an arbitrarily large number of clown faces to the database significantly increases their performance metric.
- whatshisface 7y agoBayes' theorem strikes again. Even if the odds of a false positive are only 1 in 1000, if 1 in a million are criminals then out of 1000 flagged people only 1 might be a criminal.
- mpoteat 7y agoAn acolyte of the conspiracy... Seriously though, for anyone interested in understanding this interesting statistical principle, check out the "Intuitive Explanation of Bayes Theorem" [1] [1]: https://betterexplained.com/articles/an-intuitive-and-short-explanation-of-bayes-theorem/ https://betterexplained.com/articles/an-intuitive-and-short-...
- Jorge1o1 7y agoFirst thing that came to mind
- randyrand 7y agoI feel like saying “its correct 99.9%” of the time should only said for real sample sets of input data.
- itwasntandy 7y agoIndeed. And considering the hundreds of millions[1] of people who pass through london’s stations each year, that would means hundreds of thousands of false positives. [1] https://en.m.wikipedia.org/wiki/List_of_busiest_London_Underground_stations https://en.m.wikipedia.org/wiki/List_of_busiest_London_Under...
- turk73 7y agoI have written on this exact subject beginning around 2000. Nobody freaking listened! You would think that "best minds" at the NSA and places would understand the basic statistical reality of what they are doing and why it can't work, but no, statism clouds the mind and profits cloud good judgement. Basically, if the natural error rate of a database is 5%, you aren't going to data mine a terrorist out of it if terrorists are < 1% (try .001%) of the population. Not. Gonna. Happen. What is happening is our rights are being stripped and we have a rogue FBI that manufactures criminals when it wants to grab headlines and what's more, we have PROOF that they done exactly that.
- emanreus 7y agoYep. The lack of understanding of false positives truly never ceases to amaze.
- deleted 7y ago[deleted]
- educationdata 7y ago"if 1 in a million are criminals"? NYC population is 8.6 million in 2017, so there are only 8~9 criminals in NYC? Wow. Come on, you should give a much better estimate.
- kurthr 7y agoFor each individual crime there probably is only one or a small number of perpetrators. You can't just say... hey we found 1000 (or even 5) of you face matched for a single crime (assault) which you almost certainly (or even probably) didn't commit, but after a search or warrant some of you appear to have committed a different other crime (drug possession, tax evasion, public urination, evading arrest, immigration status) so go to jail and wait for court. Once police have the right to search, enter, confiscate. and arrest probable cause is out the window.
- educationdata 7y agoI don't follow. Do they match faces to crimes or match faces to faces of criminals?
- kurthr 7y agoThey match faces out of a crowd for criminals, but to be a criminal you have to have committed a specific crime... usually that means a single person. For any one crime, they are looking for a single person and that's worst case for false positive recognition. For a large sample set 50k-1000k, they are lucky they don't match any one person to multiple criminals (birthday paradox) ... that would be obviously disqualifying. I suppose it could be worse and they just take an aggregate face of all (accused?) criminals or for particular crimes (white collar?) and match against that... hey your skin was brown, or hey you didn't shave... you appear Male.
- agumonkey 7y agoThis might be weighted with a very high risk coefficient
- jdietrich 7y ago...which makes an 81% false positive rate remarkably good. You've massively whittled down your list of potential subjects and made the job of humans immeasurably easier. An 81% false positive rate would be nightmarishly awful if this system served as judge, jury and executioner, but it's incredibly useful as a means of improving the signal-to-noise ratio of surveillance data.
- e40 7y agoThat would be true if wiggling out of the net of justice when you are innocent were not so expensive. There are a lot of poor people who can't afford the system you speak of.
- jdietrich 7y agoThe world is not America. Your criminal justice system might be a Kafkaesque nightmare, but ours is mostly OK.
- bradknowles 7y agoJust wait. If yours isn't already as bad as ours, then just wait.
- jdietrich 7y agoI'm sure that's comforting to believe, but it just isn't true. America's criminal justice system is uniquely dysfunctional in a multitude of ways, best exemplified by your extraordinarily high incarceration rate. There is no law of nature that dictates that all criminal justice systems inexorably deteriorate into tyranny, it's just a specific and fixable set of policy failures. The iniquities of pre-trial detention and cash bail are fixable. Racially biased arrest, conviction and sentencing rates are fixable. Needlessly punitive sentences and an over-reliance on incarceration are fixable. Miscarriages of justice due to pseudoscientific forensic evidence and under-funded public defenders are fixable.
- throw20102010 7y ago
- est 7y agoConviction by human court has a success rate.
- paultopia 7y ago^ this. Reasons why I teach basic Bayes to my first-year law students. If at least one of them becomes a competent defense lawyer...
- DanBC 7y agoYou might find "reckoning with risk" by here giggerenzer interesting. He presents useful tools for everyday people to understand this, mostly by converting percentages to real numbers and drawing probability trees.
- deleted 7y ago[deleted]
- cedivad 7y agoDoesn't sound too bad to me. I mean, moral implications aside this is a fairly successful technical achievement.
- arkitaip 7y agoMay I suggest that you up whatever success metrics you have because a 19% success rate is abysmal. I swear we have no quality standards in our industry, let alone strong ethics.
- benj111 7y agoAh but this is government procurement, so the fact its running at all makes it a success, and only 2 years late and 3x over budget.
- topynate 7y agoThe purpose of the system is to aid police in finding people on a wanted list. On average someone using this system only needs to look at 5 photos to spot someone, whereas without it they would need to look at thousands. The police are probably quite satisfied with the system's 19% hit rate.
- syshum 7y ago>>The police are probably quite satisfied with the system's 19% hit rate. lol, I am sure the police are, we as a society however should not be allowing the police to dictate what is and is not a "success". They have a MASSIVE conflict of interest here The police would be happy with a 1% or even less hit rate
- jdietrich 7y agoIf you had a system that picked the winning lottery numbers 19% of the time, you'd be a very wealthy man.
- TuringNYC 7y ago
- OscarCunningham 7y agoThis would be good if only we knew that they wouldn't be treated as though they're more reliable than they actually are.
- OrgNet 7y agojust keep doing it... one day it'll come down to 75%
- makerofspoons 7y agoI have yet to work with a commercial facial recognition product that produced a reasonable number of false positives or negatives when dealing with 'live' data like that of a security feed of a crowd. Is there a facial recognition model or tool out there that actually works in that use case? It seems like the Met's police got sold some technological snake oil that wasn't close to ready.
- jdoliner 7y agoIt really depends on if their goal with this was actually a tangible result or just security theater. Snake oil's not a bad purchase if you're planning to resell it.
- turk73 7y agoThe point is that it will never be "close to ready." Not unless every fourth person is a crook. Anyways, these kinds of systems exist for the same reason drug dogs exist--to generate a probable cause that appears scientific even when it is anything but.
- AnimalMuppet 7y agoHow does this compare to humans? If you have a human police officer watching a crowd and singling out "suspects", what percentage of those are innocent?
- namanyayg 7y agoNo human police offer could process these many faces or remember suspect databases, of course our current tech is going to be faster here. More inaccurate, sure, but I'm thinking that with the amount of government money involved it'll slowly improve. Or at least society will reach a stage where we don't care about the incorrectly flagged anymore.
- samlevine 7y agoThis isn't the right question. Cars help the police to move much more quickly than they could on foot, for example, and properly used can be part of a system that values justice and human rights. A better question might be: "given how accurate this facial recognition system is, what if anything in addition to a match should be required for an officer to detain someone matched by it?". I don't have an answer for this one. This tech is new, and complex, and hard. We'll likely abuse it before we figure out how to properly balance human rights and protecting the public.
- sigstoat 7y agoprecision vs recall. if it correctly finds anyone on the watch list, but they comprise only 19% of the results returned, and the police force knows it, that strikes me as a great screening tool. just need some people to sift through it. i doubt it finds the folks on the watch list that reliably though.
- gwern 7y agoOr in other words, 19% flagged were guilty. How large does it have to be to be useful? Bueller? Bueller?
- ggggtez 7y agoThe problem is that innocent people could go to jail because they fall into that 81% of incorrect matches. The justice system, at least in the USA, has a presumption of innocence. The idea being that when there isn't enough evidence, it would be better to let a criminal go free than send an innocent person to jail. I wouldn't be surprised if these systems are used specifically in cases with weak evidence. If you had phone records or other hard evidence, you wouldn't need to rely on face matching. So the faulty system ends up used in the exact cases where it can do the most damage to innocent people.
- macspoofing 7y agoNobody will go to prison because of a false positive facial detection.
- isostatic 7y agoNobody will go to prison because of a false positive DNA identification
- bradknowles 7y agoYes they will. They already have.
- isostatic 7y agosorry, my sarcasm flag must have been hidden Here's one example: https://www.manchestereveningnews.co.uk/news/greater-manchester-news/forensic-procedures-criticised-as-man-wrongly-799500 https://www.manchestereveningnews.co.uk/news/greater-manches... "Mr Scott, from Truro in Cornwall, spend nearly three months on remand in Strangeways after being charged with a single count of rape. He strongly protested his innocence - and told detectives in interviews that he had never been to Manchester. The Crown Prosecution Service formerly dropped the case during a hearing at Manchester Crown Court after the private firm confirmed the DNA blunder. " Despite warnings of DNA contamination at the laboratory less than two week earlier, Adam Scott was arrested and held in custody – even though he had been hundreds of miles from the scene of the crime. https://www.independent.co.uk/news/uk/crime/rape-accused-adam-scott-was-victim-of-forensics-error-regulator-finds-8193163.html https://www.independent.co.uk/news/uk/crime/rape-accused-ada...
- ggggtez 7y agoPut another way: they are saying that 81% of positive flags are actually false positives. Yikes. That might be better than random chance, but in the court of law that should hardly be considered evidence.
- LudwigNagasena 7y agoI think even 95% would be good enough, 81% is commendable.
- kd5bjo 7y agoThere aren’t enough details here to tell whether or not this system is truly useful. If its false negative rate is good and other methods successfully detect the false positives, like traditional detective work, using it as a screening tool reduces the number of people that need to be investigated by a factor of 200 or so^. However, those are a lot of assumptions— there’s lots of ways this could actually be as bad as it looks on the surface: - Is the output of this system being used as evidence in a courtroom instead of a lead generator? - Is there some systematic bias in the false positives that result in a particular class of innocent people being harassed by the police during their investigations? - Does the system reliably include the actual suspect in its set of potential suspects? I’ll refrain from commenting on the social acceptability of this system, as I have never been a part of UK society and thus lack a baseline against which to compare it. ^My brain is refusing to do the arithmetic tonight for some reason.
- fencepost 7y agoThe panopticon is coming and its accuracy will improve. Facial recognition by itself may not improve much, but when paired with other information it's going to skyrocket in accuracy, and I suspect it'll happen in China first if it hasn't already. That other information will include things like the possible identities of other people in the same vicinity that have known associations, locations pulled from recognized devices, other fuzzy identifications along travel routes, gait analysis, matching facial recognitions with clothing identification to improve tracking, etc. None of the individual parts are that difficult, they just result in a massive amount of fuzzy data to be mined and computers can be very good at that mining. Another factor is that this may be challenging when starting from a facial recognition point in a crowd and trying to identify a person, but what if you're starting from a list of "the usual suspects" who've been thoroughly photographed, had gait analysis done and had their social networks mapped? What if you start from "do we see any sign that any of these 50k individuals are in the crowd?"
- heavenlyblue 7y agoWhat if you’re breaking the privacy laws doing so? Marijuana is illegal, so human tracking should be illegal too.
- bradknowles 7y agoWhat privacy laws? You thought you ever had privacy?
- heavenlyblue 7y agoBut I did before. Now you’re just inventing new ways of policing... - yet policing what, exactly? The crime levels are at the lowest points in the civilised countries.
- stordoff 7y ago> Marijuana is illegal, so human tracking should be illegal too. I'm not sure I really see the connection here.
- I_am_tiberius 7y agoAlso if this technology works perfectly I don't want to be checked everywhere. It should be a human right to be excluded from facial recognition checks.
- Animats 7y agoWe need is a "you only get hassled once" system and policy. If you're stopped by the cops for looking suspicious, but not arrested and convicted, you should get a goodguy tag that prevents you from being stopped for a few years.
- 24gttghh 7y agoOr we could not do this at all in the first place and tell the authorities to go pound sand.
- stordoff 7y agoThat seems open to abuse - wander around London until you get pinged with a false positive (vary your appearance - hat/glasses/beard - to make it more likely), then go and commit your crime, knowing one of the major tools is neutralised. Either it's good enough for general use, and we don't need this (_maybe_ have an exception for people who have been hassled multiple times), or it shouldn't be used.
- jimclegg 7y agoRemeber the black guy with no record in Florida that was stopped 20-30 times a week, making his life unlivsble (hard to keep a job). His life is going to get a lot more interesting with technology like this, since we know the false positives tend to have a specific race.
- deogeo 7y agoYet another article framing dystopian levels of surveillance as only bad if they malfunction. As if the only reason we wouldn't want to give complete control over our lives to authority, is because the authority might make mistakes.
- magicalist 7y ago> Yet another article framing dystopian levels of surveillance as only bad if they malfunction Seriously, it's amazing how well framing a conversation works to divert it. Where are all the comments about the stasi, securing databases, corrupt law enforcement, and not collecting information in the first place? Instead we're arguing over what percentage of "innocent" people can be hassled by the total surveillance state for it to be acceptable.
- ltbarcly3 7y agoIt's only dystopian if we end up in an actual dystopia. Right now this is inconveniencing people. There is a totally valid question of whether we want surveillance at this level, and whether it is giving too much power to the government, which may not always use it for valid or good reasons. I don't support at all the use of mass surveillance because it's too easy to abuse, but in a world where we carry a location beacon with us everywhere we go, I don't know if we lose much if there are cameras in public places?
- darawk 7y agoReducing the search space from everyone in a crowd to 5 people seems...pretty great technology wise. I'm not sure what the problem is here. I mean, I get that people don't like facial recognition for privacy reasons. But that seems totally orthogonal to this report, which is criticizing its efficacy, on very odd grounds.
- mikeash 7y agoI totally agree, if the users treat it accordingly. The trouble is when the police assume there’s a 99.9% chance that each flagged person is guilty because the system has a 0.1% false positive rate.
- mola 7y agoWhen you are on of a crowd, you don't get treated like a suspect. When you are 1 of five you get treated like a suspect. A criminal. That direct implications on your well being. 4 people out of 5 will be Innocents that get treated like criminals. I don't want to live in that world.
- sysbin 7y agoWe're already living in a worse world than that.
- darawk 7y agoOk...but that happens all the time. If you look like someone that just robbed a liquor store and you're in the area, cops will stop you until they can clear you as a suspect. This is how policing has always worked.
- duxup 7y agoIs a dragnet where everyone in a given area is checked even all that common? Seems unlikely. But with technology we can suddenly do it 24/7, numerous places, and for all sorts of crimes at the push of a button...the implications of that many people as suspects, that volume of innocent people accused seems pretty bad.
- rndgermandude 7y agoBerlin did experiment with facial recognition. They had 300 test subjects that regularly used a particular train station. They claimed to have achieved a 65.8% detection rate, or up to 200 properly identified persons per day (so 1/3 were false negatives). However they also identified 0.67% wrongly, so given 90,000 people using that station a day, that means about 600 people were wrongly accused by the system each day (false positives). They declared the system, which wrongly suspected up to 600 people a day at a single train station of being wanted criminals, but missed a lot of actually wanted criminals too, a major success. Mind you, they used high resolution photographs as the training input for the system, something police does not necessarily have available of criminals and suspects they try to track down. https://www.ccc.de/de/updates/2018/debakel-am-suedkreuz https://www.ccc.de/de/updates/2018/debakel-am-suedkreuz (German)
- e40 7y agoRecently, I've thought of the German government more the champion of individual rights than the US, but this just shows how far down the path of zero-fucks given about the damage these systems can do to people.
- rndgermandude 7y agoAre we talking about the German government that was pushing mandatory communication meta data retention for all? That classifies people as "dangerous" without a trial and allows for the temporary imprisonment of those people without any court involvement whatsoever? The government that was a driving force behind the copyright directive (aka article 13)? The government that made unconstitutional laws to allow police to install trojans on people's devices? That tried to push an EU-wide censorship infrastructure with secret filter lists under the guise of "fighting child porn", while lying about child porn (e.g. they claimed child porn is not illegal in India, which is not true) and where then-family minister von der Leyen (now about to become EU commission president) screened actual child porn to journalists at a press conference to influence their reporting? The government that changed the law so that now you have to talk to the police most of the time (as a "witness") That helped the NSA to spy on communication of other EU leaders and German politicians as well? And then tried to hide it when caught, using every dirty (legal) trick imaginable? That regularly does mass surveillance on peaceful protesters? Or let the police lose to bash up a few of them? That made a law that essentially turned social media companies into law enforcement agencies with no supervision? That currently tries to create laws to force service providers to turn over communication "even if it is encrypted", aka defacto crypto backdoors? and so on ...
- jacquesm 7y agoProponents of this system will chalk this up as proof the system works because 19% of the suspects were not.
- onetimemanytime 7y agoso 19% were criminals :)
- keiru 7y ago1% error would arguably be scarier for that 1%, just pointing it out. I hope this tech evolves along some strong regulations on how they are used as evidence.
- kazinator 7y ago19% of suspects flagged by Met's police facial recognition technology are not innocent. That's pretty darn useful!
- xmichael999 7y agoRight, I made this point kinda of in another comment. It really depends on how you want to frame the conversation. I agree with the general argument that randomly stopping and questioning people because a computer hit sucks, but knowing UK law enforcement they likely politely stopped the suspects and asked to see ID. Once they provided ID they were free to go on about their business. This technology is catching a ton of people that would otherwise not be caught.
- mantap 7y agoAnd what if you don't carry ID, as is normal in UK. Do they arrest you?
- kazinator 7y agoSuppose your face is identified by a machine as looking like someone in a criminal database who is wanted. Suppose that you in fact are a dead ringer for that person in the database; even to a humans who double check this match, you look exactly the same. You have no ID. What are they to do?
- jammygit 7y agoThis is a temporary state of things and people need to be less caught up with the technical shortcomings. It’s the world where machine vision is fully accurate that is the real problem, not only the one where the machines are buggy
- tracker1 7y agoSo the machines think all <insert-identity-group> look alike.
- xmichael999 7y agoOn the surface this sounds bad, real bad. However on the flip side it means they found a number of criminals they would have never found otherwise. Hopefully they can tighten the false positive ratio dramatically but this technology for all its faults is finding real criminals that would otherwise not have been found.
- TuringNYC 7y agoYour having already labeled those found as “criminals” rather than “suspects” speaks volumes to the problem with legitimizing such systems.
- bb88 7y agoFor every criminal found that means 4 innocent people are flagged. Hence the 80% rate. But if there are 1 in 10,000 people which are criminals, then 5 total people out of 10,000 will be flagged. So the chance of being flagged would be 5 in 10,000 or 1 in 2,000 which would be about a .05% false positive rate overall. I think everyone forgets about in a situation like this is the false negative rate -- or missing a flagged person. If they miss 90% of the people they flag, then the system is pretty much worthless.
- ineedasername 7y agoThe article doesn't really go into enough depth on the repercussions here one way or another: -What is the level of impact on the individuals caught up in a false positive? A minor inconvenience? A major issue? -It takes resources to deal with false positives, even 1 in a 1000 can be a lot of people. The question isn't how much it costs though, it's how much does it cost without the automated facial match, and how much to the false positives cost, and what is the difference between them?
- Haga 7y agoHe has the jaw of a criminal, the brows of a mass murderer and the news of a crook. physiognomie nn, the automated racistic quack science, brought to you by the valley.