7 ms·
A new way police are surveilling: Calculating threat ‘score’
- leke 11y agoIt's like a high-tech version of profiling.
- griffinmb 11y agoIt reminds me of Psycho-Pass. "Psycho-Pass is set in 2113. The Sibyl System (シビュラシステム Shibyura Shisutemu?) is actively measuring the populace's mental states, personalities, and the probability that individuals will commit crimes, using a "cymatic scan" of the brain. The resulting assessment is called a Psycho-Pass (サイコパス Saikopasu?). When the probability of a person engaging in crimes measured by the Crime Coefficient (犯罪係数 Hanzaikeisū?) index exceeds a certain level in an individual, he or she is pursued, apprehended, and killed if necessary." Taken from: https://en.wikipedia.org/wiki/Psycho-Pass https://en.wikipedia.org/wiki/Psycho-Pass
- Paul_S 11y agoAs plausible as Minority Report. Even if possible it would not be implemented because of how the majority of people think of human free will.
- magoghm 11y agoI also was reminded of Psycho-Pass.
- pc86 11y agoExcept this isn't a brain scan in anime. It's basically a mathematical model based on prior criminal history, gang affiliation, etc.
- habitue 11y agoThis is interesting since, really, cops' brains are already creating "threat scores", it's just that they aren't available to look at or tinker with how they're calculated. It's likely these computer models are inferior to the mental models, but making them explicit is a big step.
- mordocai 11y agoIt would be a big step if those models were under public scrutiny and not a "trade secret" of the for profit company that makes it.
- habitue 11y agoFair point. But it's now also possible to lobby to get access to it
- xiaoma 11y agoAs soon as the model were public it would be useless. The first thing an aspiring criminal or terrorist would do would be study the model and then do whatever necessary to make his or her own threat score low.
- elevation 11y ago"The program scoured billions of data points, including arrest reports, property records, commercial databases, deep Web searches and the man’s social- media postings." Does "deep Web searches" mean the program has access to your tor activity? How is that information acquired?
- kragen 11y agoNo, the "deep Web" is things like friends-of-friends Facebook posts, Twitter posts to locked accounts, and friends-locked LiveJournal posts — things that are in some sense "on the web" but that a standard search engine can't find. You're thinking of the "dark Web", which includes things like onion sites. Your confusion is not unusual; https://en.wikipedia.org/wiki/Dark_web https://en.wikipedia.org/wiki/Dark_web says, "The dark web has often been confused with the deep web, the parts of the web not searched by search engines. This confusion dates to at least 2009."
- jMyles 11y agoTurning the profiles and prejudices of police into an explicit metric, whose code can be examined and debated, seems like an idea that might have not have legs in a free society. However, surprisingly unmentioned here is the inevitability, so long as the client is the state, that these metrics include political dissent as a threat model. Has anyone asked Beware for comment on this?
- cryoshon 11y agoThis is probably the unstated purpose of the software rather than providing stratagems for officers responding to incidents-- the repertoire of possible moves contains only "violence" or "ultraviolence", so information on which to use when responding to an incident isn't that useful. The police already build dossiers on people who attend protests so that they will have targets to discredit or harass. When do the citizens start building dossiers on the police?
- newman314 11y agoI'm definitely for scoring the police if they insist on rolling this out. Police should not be above the law. Unfortunately, this is not likely to happen just like LEO are able to hide their home address because safety but you Joe citizen do not have that option.
- sliverstorm 11y agoIf political dissent is correlated to violence, why shouldn't it be a parameter in the threat model? Everyone, including police, form threat models in their own minds all the time. Ever avoided the hoodied skinhead hiding his hands in his pockets in a dark alley? Threat model. The model isn't grounds for prosecution, but it is reasonable grounds to adjust your response.
- pjc50 11y agoIf political dissent is correlated to violence, why shouldn't it be a parameter in the threat model? Because we know this will be implemented in a partisan way. "Dissent" implies that there is a single politics from which everything different must be considered suspect. Simply using the word implies totalitarianism. You can see this in the US where the white people talking about the violent overthrow of the government somehow aren't considered terrorists and are relatively unmolested even if they actually occupy Federal land and buildings, but a nonwhite kid holding a toy gun gets shot in a playground. Edit: just to drive the point home, does the threat model include a database of gun owners? Do we think gun ownership might be correlated with crime? Or is it illegal to investigate that as it is for the CDC?
- ck2 11y agoOh wow, they probably just gave equifax a truly horrible idea. I wonder if this is even legal since it creates prejudice, probably is through some loophole. Did you ever protest a war? Ah you have a terrible threat score. Next they open it up to employers.
- intopieces 11y ago>Fresno Police Chief Jerry Dyer said officers are often working on scant or even inaccurate information when they respond to calls, so Beware and the Real Time Crime Center give them a sense of what may be behind the next door. This assumes that the information present on the Internet is accurate, a scary thought. How difficult would it be for someone to make a profile with your first and last name on any or all of the major social networks, painting you as a gun nut, a recovering drug addict, or worse? Allowing this system to inform police work without the least bit of public oversight is irresponsible. But that won't become a concern, I suspect, until something unfortunate happens. For now, the only stories we have about this technology's use point to the 'bad guys' being caught.
- gregmac 11y ago> But that won't become a concern, I suspect, until something unfortunate happens. And then it will just be dismissed as an unfortunate side effect, shadowed by all the times the system worked and did good. After all, isn't it worthwhile to have a SWAT team bust down a few doors and maybe kill one innocent person if it helps catch hundreds of 'bad guys'? (/sarcasm, in case it isn't obvious)
- wfo 11y agoI think the idea of someone fabricating legal-but-indicitave online information about you so that you turn red in the police's system and then setting you up to be considered a threat in an incident where the police assume you are dangerous but you are not and are goaded into shooting you is kind of a stretch. Maybe a good plot for a movie. But the program itself is somewhat troubling. But right now we have a profiling system the police use. It's 'dark skin -> almost certainly has a gun' instead of 'BEWARE says green BEWARE says red'. Is looking at public info with an explicit review-able algorithm better or worse than letting officers profile naturally? I'd agree using data like this for police purposes can be fraught with issues and is troubling but even as a privacy advocate this system seems at the very least not worse than what we currently do. Not because it is good, but because what we currently do is so bad.
- mordocai 11y ago
- Nadya 11y agoAh yes, more propaganda supporting a surveillance state. What makes these sorts of things difficult to argue about is all a matter of how they get framed. "It's good for a more peaceful society. You aren't violent? Right? You have nothing to worry about then! This technology will help keep you safe. You want to be safe? Right?" I'm glad to see many citizens are at least sharing their voice against this. But it reads all to much like Fresno took "big strides too fast" and if they had taken baby-steps there would have been less outrage. You have to whittle down people's liberties - not try and take them all at once! If you do it slowly enough, they don't even notice. As an aside... typo + how do they know their score? I thought only operators knew it? How does she know her threat level was elevated because she tweeted about "Rage"? >Once council member referred to a local media report saying that a woman’s threat level was elevated because she was tweeting about a card game titled “Rage,” which could be a keyword in Beware’s assessment of social media.
- deleted 11y ago[deleted]
- deleted 11y ago[deleted]
- acqq 11y agoWikipedia to the rescue: https://en.wikipedia.org/wiki/Statistics_of_incarcerated_African-American_males https://en.wikipedia.org/wiki/Statistics_of_incarcerated_Afr... The first graph looks biased (or unclear) in the representation (percent of what?), but the PDF used is here: http://www.bjs.gov/content/pub/pdf/pim09st.pdf http://www.bjs.gov/content/pub/pdf/pim09st.pdf The total of males in prisons in 2009: 2,096,300 White 693,800 Black 841,000 Hispanic 442,000
- Paul_S 11y agoWhat was in the (now deleted) post you were replying to?
- acqq 11y agoAs much as I'm able to repeat it correctly, the author wrote that he believed that the information about the ethnicity will be unjustly involved in the algorithms. Once being aware of the statistics, there's also: https://www.youtube.com/watch?v=8ABRlWybBqM https://www.youtube.com/watch?v=8ABRlWybBqM
- Paul_S 11y agoReminds me of the video of a woman beating up a guy on a street to the amusement of the onlookers. https://www.youtube.com/watch?v=u3PgH86OyEM https://www.youtube.com/watch?v=u3PgH86OyEM
- hackuser 11y agoAlso, police in at least some cities use systems that predict whether someone will become a criminal and give that person extra attention. Here's one story on it that I found with a quick search: http://www.nytimes.com/2015/09/25/us/police-program-aims-to-pinpoint-those-most-likely-to-commit-crimes.html http://www.nytimes.com/2015/09/25/us/police-program-aims-to-...
- ryandrake 11y agoWow, they're even calling it "Predictive Policing". Here's my prediction: By 2020, we will have fully automated arrest warrants being served and executed based solely on big data analysis, and the legality of that practice will be working its way through the courts.
- oxryly1 11y agoPrecog Unit.
- hackuser 11y agoA few considerations: 1) If these are good ideas and there is nothing wrong with them, than why does law enforcement keep them secret? Why don't they advertise to citizens: 'We track everything you do, including who you call, where you go, what you say online, and then we predict if you are a threat or not and respond appropriately. This makes you and the community safer!' 2) Does being black or Muslim, having far-left or far-right political views, increase or decrease your threat score? 3) Law enforcement has a very well-established history of using their power not for public safety, but to abuse and oppress vulnerable classes of people and people politically unpopular with the state (e.g., peace protesters). Just think of all the shootings, arrests, COINTELPRO, etc. etc. Why would this power be used any differently? For example, many blacks already experience harassmet from law enforcement, including such things as pulling them over for 'driving while black', suspecting them of criminality for being in a predominently white shopping area, arresting them for no cause, etc. What will life be like for blacks on a police threat list? Will their life be constant harassment? Will the predicted threat be self-fulfilling because they will be arrested inevitably? Will corrupt police just plant something on citizens on the threat list and arrest them to preemptively get them off the street (and then tout the accuracy and value of the threat list)? 4) Police already are given the authority to shoot people based on suspicion or a feeling of threat. For example, someone putting their hand in their pocket or reaching for something in their car could be reaching for a gun. Will being on the threat list be justification for violence against citizens? 5) Aren't we presumed innocent until proven guilty? Aren't we to be judged by our actions and not what someone else thinks we will do? What happened to the land of opportunity?
- kragen 11y ago> Does being black or Muslim, having far-left or far-right political views, increase or decrease your threat score? One of the things they explicitly mentioned was "gang associations". In my US experience, groups of black or Hispanic* youths who sometimes do illegal things (shoplifting, trespassing, underage drinking, pot smoking) are often called "gangs" in cases where groups of white or Asian youths wouldn't, even if they were doing the same things. So I think it's inevitable that this kind of Citizenship Score will reproduce and fortify existing structures of racist discrimination. * This primarily applies in parts of the US where the local system of racism classifies Hispanic people as nonwhite.
- SN76477 11y agoReminds me of the Sesame Score that was being talked about so much a few weeks ago. The Mark of the Beast looks more and more real every day it seems.
- jrcii 11y agoThey must have read this article https://www.aclu.org/blog/free-future/chinas-nightmarish-citizen-scores-are-warning-americans https://www.aclu.org/blog/free-future/chinas-nightmarish-cit... and misread "warning" as "inspiration"
- philfrasty 11y agoReminds me exactly of the intro from „Person of interest“ https://www.youtube.com/watch?v=WOnQ8CD3v4g https://www.youtube.com/watch?v=WOnQ8CD3v4g
- wj 11y agoIt also is the central plot point to the anime Psycho Pass. (off-topic but the watch in Psycho Pass was what I hoped the Apple Watch would be)
- jcromartie 11y agoWhy shouldn't we use statistical inference to deliver information that enables responders to be better prepared to deal with a situation? I would hate to be a false positive, but I'd rather have the chance of being flagged as "green" when police are headed my way, rather than always being presumed to be a "red" as seems to be the case now. This seems like it would be the opposite of being presumed guilty until proven innocent, for most people. And this isn't surveillance: they aren't collecting new information, just putting available data sources together. Unless deducing things from existing data is somehow surveillance?
- noja 11y agoThe issue is judging everyone before a trial. i.e. the opposite of the presumption of innocence PLUS huge scale. Like mass tracking of people using phones versus being tailed.
- lukeschlather 11y agoThe anecdote from the article strikes me as a really good example of why this sort of statistical profiling is unnecessary. Guy threatens his girlfriend, algorithm flags him as red, so they... call in a negotiator and successfully de-escalate the situation. I'm struggling to see how the system flagging him as red had anything to do with their actions. It was just a case of cool heads doing good police work.
- VLM 11y agoCops already have a lot of contact with parole inmates who have dramatically reduced rights. So they'll just lump "red" in with the parole inmates. You'll hear cops saying things to each other like "he wouldn't be red unless he deserved it". Generally, having the cops prejudiced against someone before they even meet them is not going to turn out well.
- newman314 11y agoUS cops are not trained to prioritize descalatio in general. That's why you have that cop in McKinney doing a diving roll like Farva.
- deleted 11y ago[deleted]
- iamleppert 11y agoYou know I hate to admit it, but as techies we are responsible for most of this power that the police have. We have created all this technology, and then some business has gone into local police departments and pitched the idea. Kind of sad to realize there are people in our industry who don't mind doing bad things with their talents or are at best indifferent to what their company is doing. It's the "I just work here" attitude so you don't have to care if the company has an actual business model, helps people, creates a good product, or is even doing something ethical or not.
- roflchoppa 11y agoIt appears that they are only using it as a caution to ensure that the responding officers have the resources so that the public, and the offices are not in harm. Now as far as the aggregation of publicly available posts to the internet, as well as corresponding that to private-state information (previous records / offences). Its nice that SAAS today can be updated on the fly to ensure that the threat scores can become more refined, so in the example of the Yellow House score for the council member, might have been avoided.... unless hes actually a yellow based on his web traffic..... I wonder if they apply this to pulling people over as well. Think, is the car reported stolen (alert: RED), if not then the highest color for the house hold. But lets also keep in mind that Fresno Ca, is not Palo Alto, Ca.
- cgearhart 11y agoThe scariest part is that this kind of program drops all pretense and explicitly characterizes all interactions with the authorities in the context of the "threat" each of us poses. It anchors the discussion in terms inherently antagonistic relationship, and normalizes aggressive policing behavior. Why aren't more people scared by this? Although it is inconvenient for law enforcement, the government should not conduct persistent, universal surveillance on the citizenry - even if it only consists of monitoring & compiling publicly available information. The value of these tools is dubious, and the potential for abuse is high; there is no accountability for the costs in comparison to the utility, and they are routinely established in the absence of oversight by design. Mechanisms already exist for targeted surveillance when authorities have probable cause. Normalizing casual surveillance with no specific intent marginalizes the discussion of more invasive surveillance techniques, and emboldens authorities who dismiss privacy as a trivial impediment to security.
- dmix 11y agoThought experiment: is this very different from police targeting high-risk neighbourhoods? Police have been over-policing black and poor neighbourhoods for decades but no one is protesting that. There will certainly be a correlation between "interactions with the authorities" <-> where police spend their time patrolling, skewing the data. The only time I've ever been stopped and interrogated randomly, simply for walking down the sidewalk with a hoodie on (in winter) in a bad neighbourhood, was visiting a friend in a housing project. Asking where I was going and who am I meeting. All of his family and friends get harassed by police living there, despite having zero background of criminality. And in Toronto they record your name/location each time you get stopped and carded in an "anti-gang" database.
- pjc50 11y agoPolice have been over-policing black and poor neighbourhoods for decades but no one is protesting that. Er, apart from all those Black Lives Matter protesters that keep getting in the news?
- 11y ago
- joshbuddy 11y agoImagine that this is just employing machine learning, where the training set is police reports filled out after the fact. Is this just big data being applied to threat prediction?
- VLM 11y agoNote that there's not just shoot to kill response. They specifically mention "property records". For me, that works out pretty well, I have enough money to afford enough property to indicate I can easily afford legal representation and will be treated as such by the police, yes mr vlm, no mr vlm, you are free to go mr vlm, have a nice day mr vlm. On the other hand, people verified to be dirt poor can expect dramatically different treatment knowing they cannot afford legal representation. On the downside I can expect no mercy WRT ticket fees, because they'll know I can afford it. You can envision traffic cameras lighting up when I drive thru a speed trap village, between my age, clean criminal record, and property ownership they'll know they can collect quite a toll off me at zero personal risk. Aside from property, anyone who's ever posted anything vaguely drug or alcohol related on the internet, ever, now has perpetual probable cause for a breathalyzer test or a drug test down at the hospital. This is perfect for police harassment.
- fapjacks 11y agoSo... Here's my thoughts on it. At first blush how dare they! But as I let this stew for a while and turned it over a few times, I'm arriving at a different conclusion, helped along by some of the insightful comments here. Since the police really are doing this kind of thing in their heads anyways, what I see here is an opportunity for accountability. Did the man in body armor with the gun consult the magic number before he shot the kid on the playground with a toy gun? Or did he just get excited? Like body cams, I think this is actually something that could be used in favor of a more responsible police force than not.
- ErikVandeWater 11y agoTechnology moves too fast for the law. Even if police believe using a particular technology is unconstitutional, they are likely to use it anyway, because it will likely take years for a challenge to be brought to court, and definitely years to get to the highest (influential) courts. Even after all this, the police will only have to stop the unconstitutional activity, and immunity will handle the rest. States need to enact laws that will prevent police departments from using any new software without prior approval of their constituents (e.g. city council approval).
- kevinSuttle 11y agoThe scary thing is, this is pretty much the plot for Captain America: The Winter Soldier. http://www.imdb.com/title/tt1843866/synopsis?ref_=tt_stry_pl http://www.imdb.com/title/tt1843866/synopsis?ref_=tt_stry_pl
- newman314 11y agoI'll also point out that this is inherently unfair fron the standpoint using a person's name as the primary key for lookup. Given the roaring success that is the no fly list (/s), people's names are not evenly distributed, there are a lot more John Smiths than say Larry Ellison. It should not mean one person is easier to identify because of their name. If that's the case, there should be an even name distribution to give everyone a fair shake. Let's call everyone Bob!