12 ms·
Palantir deployed a predictive policing system in New Orleans
- dmckeon 9y agoIs "network analysis" a better term for what this article describes? Seriously, if taking a deep, hard look at connections among alleged drug dealers and gang members is bad, what would be better?
- mulmen 9y agoThe word "alleged" certainly concerns me w.r.t state surveillance, especially in the hands of law enforcement. Connecting dots between convicted criminals is one thing, predicting crime based on associations with people who are not criminals is a whole other ballgame.
- derefr 9y agoA predictive model isn’t always trying to predict the future. It can also be used to predict the present—to find hidden connections in (perhaps maliciously) incomplete data. In this case, that’s to say: if an unnamed gang member committed a crime, but all the known members of said gang have alibis... maybe you should be looking for an unknown member of said gang. And who’s likely to be secretly in a gang? Well, someone whose friends are all in said gang would be a good first guess.
- DoreenMichele 9y agoOMG. What about outreach workers? How many of them are going to have connections to junkies, homeless people and gang members? Sincere good Christians often believe in trying to help the unfortunate. Some good Christians spend all their time surrounded by clean living church goers. Others spend a whole lot of time surrounded by folks who are from the wrong side of town and the wrong side of the law.
- ggg9990 9y agoThe point of these analyses isn’t to say “arrest this man he’s a criminal” but rather “closely observe this man and ascertain whether he’s a gangbanger or an outreach worker.” Of course it doesn’t even need to get that far since it’s trivial to tell the two apart from the network analysis alone (the outreach worker will have a far higher ratio of legitimate-world contacts to criminal-world contacts than the gangbanger).
- sneak 9y ago“Have the police scrutinize the outliers” is a fantastic policy if you are fine building a society that can’t support building a community like HN.
- deleted 9y ago[deleted]
- deleted 9y ago[deleted]
- skate22 9y agoSeems like an unfair assumption. "Legitimate-world contacts" could be criminals too.
- alistairSH 9y agoAnd you don't think law enforcement agencies will quickly make the leap from "closely observe" to "harass and arrest for any tiny transgression in an effort to coerce a confession"?
- Chriky 9y agoI don't, no, why would they? --- Downvoters, care to explain?
- deleted 9y ago[deleted]
- qubex 9y agoThe word ’predict’ gets used a lot and is sufficiently charged that it causes much Minority Report-style angst and gnashing of teeth, but really a better term would be ’uncover’. The whole investigative process is about uncovering concealed connections between people and events, so this is just a new technique. It isn’t inherently Orwellian either: there’s still a wide chasm of human investigators, prosecutors, judges, and juries between any kind of ’positive’ and actual consequences.
- x12222 9y agoHave you been an innocent target? Otherwise you are just repeating conformist nonsense like "everyone knows it isn't that bad".
- farnsworthy 9y agoI was going to suggest something, but decided I'd better read the article first. And they simply said it themselves: "“We’re kind of a prototype,” said Matalin. “Unless you’re the cousin of some drug dealer that went bad, you’re going to be okay.”" Sorry, cousin! Took your house and car too, I see...
- leroy_masochist 9y agoI think the issue is whether the improvements in both deep analysis and real-time situational awareness constitute an infringement on personal liberty. My understanding of the Palantir law enforcement use case is that, for example, a gang unit police cruiser going down the street in a bad neighborhood can pick up data from its license plate readers, and be automatically alerted if any passing vehicles are associated with certain parameters. The plain-vanilla tech solution for law enforcement would be something along the lines of: throw a notification if any of the passing vehicles is registered to someone with outstanding warrants. Palantir takes it a step further and throws a notification if, say, the vehicle is registered to a known associate of a gang member who is wanted for questioning. This network understanding has to be built on the backend, both by integrating existing data sources and by doing work directly on the Palantir system. Eventually the cops might start following the vehicle, and pull the driver over for a nominal traffic infraction. At that point, the officer is going to use their Palantir app within their department-issued mobile phone to assist in the discussion with the driver -- for example, if the driver coughs up any names, the officer can type them in to see what comes up in the system. I'd think that Palantir's defense of this would be something along the lines of, "everyone in this story previously had the right to see every bit of information the system provided, we just made it more efficient". The counterargument would be something along the lines of, the state having such detailed understanding of its citizens' personal lives is ipso facto tyranny.
- jessriedel 9y agoThe ability of the government to temporarily detain and question essentially anyone at any time through traffic stops for minor violations seems like a bigger problem. It seems weird to try and restrain abuse of that power by keeping the police under-informed rather than just eliminating that power directly.
- vorpalhex 9y agoThere are defenses against this, but they're not reasonably in grasp for most people. If I start having a police officer interrogate me about something outside of a traffic stop: 1. I know enough to stop answering questions 2. I have a lawyer I can call and ask to get involved Both 1 and 2 are, however, not choices for the vast majority of people.
- vorpalhex 9y agoI'm going to talk about a counter case that happened in real life, where a journalist was put on a "kill list" by the US Government. If you haven't read the entire [Drone Papers](https://theintercept.com/drone-papers/ https://theintercept.com/drone-papers/) I highly recommend doing so. The government basically had a naive likelihood analysis program that attempted to group terrorists. If someone went to the same places as terrorists, in about the same schedule, then the system came to the conclusion they were a terrorist. In programming we use the duck metaphor - "if it walks like a duck and quacks like a duck, it's probably a duck." What occurred was that a journalist who was interviewing these terrorists ended up on the kill list. In theory the human element was supposed to filter out such things - and I'm sure that's what the developers of the system intentioned. It's also clear the human operators were simply rubber stamping everything the system produced, and allowed a kill order on a journalist to go out. And if you're discussing evaluating types, then our duck solution is just fine. If you're discussing firing hellfire missiles at unarmed people, then we probably need to take a step back. Predictive algorithms and network analysis are naive - they're fine for recognizing pictures of dogs and cats, or pinging a friend when you upload a photo with their face. They're not sufficient to apply to policing or resourcing, especially since algorithms display the exact same bias as their input data. System designers, including myself, often expect human elements to act as reliably as algorithmic components. The truth is that humans are lazy, biased and easily prone to pavlovian conditioning (The system showed me a dialog, when I click OK the thing I want to happen happens.) The other truth is that frequently these systems, even if they are honest - they show likelihood percents, their flaws have been explained to their operators, they were made with the best available data - are too complicated for most people to grasp. Most folks can't schedule a meeting across five calendars without assistance or get email marketing correct. Do we really expect them to grasp the nuances of a statistical network analysis model and correctly interpret the results on a repeated basis? The solution is that we need to stop pretending an algorithm is a replacement for human decision making in these life or death cases.
- wfo 9y agoWell first of all, any time a private corporation does secret private proprietary unreviewable analysis on members of the public which is then used by law enforcement -- this is a terrifying, horrible dystopian miscarriage of justice on its own. People (i.e. the politicians who signed off on this) should go to jail for doing this. Police should be accountable to the public, allowing them to dodge this by using private entities who are not accountable to do their dirty work is quite simply criminal corruption. Add on top of that it's being done by a company named after a mass surveillance device used for evil in a fantasy story. And on top of that it's being done by a Thiel company. Thiel, who is nearly a perfect personification of evil: he has made very explicit candid public statements on how he opposes the idea of democracy itself, he does not think women should have the right to vote. And on top of that it was being done without the knowledge or consent of nearly anyone in the city. And finally, the justice system presumes innocence. The Palantir system does the opposite -- it makes wild, arbitrary untrackable inferences that suggest guilt for people without any real evidence. It is quite literally nothing more than a very thin shield police can use to justify harassing and intimidating the "kind of people who tend to be criminals", which in this case is not that at all -- it is "the kind of people who tend to get caught and prosecuted for crimes" i.e. only violent or drug crimes (except drugs white people use), only poor criminals, only minority criminals. Where's the massive computer analysis system that looks for wage theft committed by employers? This is after all how the vast majority of wealth is stolen in the US, citizens could recover billions of their own money if it were stopped. Where's the computer system working with law enforcement to automatically detect any insider trading? Why don't we monitor the behavior of people in finance to detect cocaine use and then send in the SWAT teams? What about a computer system that detects bad prosecutors?
- hackerman12345 9y agoYou are misreading his comments about women and democracy. He was simply saying that certain demographics aren't receptive to libertarians - he was not saying that women shouldn't have the right to vote and that democracy is bad. I daresay it's impossible to read his full writing in question and come to the conclusion one might get from simply reading an excerpt of two sentences that the Politico article wants you to see. " It would be absurd to suggest that women’s votes will be taken away or that this would solve the political problems that vex us." "I believe that politics is way too intense. That’s why I’m a libertarian. Politics gets people angry, destroys relationships, and polarizes peoples’ vision: the world is us versus them; good people versus the other."
- anigbrowl 9y agoPer one of the project's boosters: “Unless you’re the cousin of some drug dealer that went bad, you’re going to be okay.” Is that really the standard you want to rely on - that you're a suspect because of what your cousin may have done?
- omarforgotpwd 9y agoNice, they mentioned the company I founded in college, PredPol. They claim that our algorithm encouraged over policing of minority neighborhoods, but there's no such thing as bad press right?
- deleted 9y ago[deleted]
- jrs95 9y agoMaybe there’s such thing as bad downvotes either, your light gray text just draws attention to your comment. I’m kind of interested as to what the definition of “over-police” is in this context, though. Is it disproportionate with or without the consideration of crime levels? It seems like over correcting for this problem could be very bad for the communities it’s intended to protect.
- omarforgotpwd 9y agoPeople who are making conjectures about the effects of the app really do not understand it or how it was used at all. It's reach and impact were far more limited than most people seem to assume.
- CompanionCuuube 9y agoThe light gray text did draw my attention to the informed opinion of someone who has worked in that business space.
- snsr 9y ago> independent academics found it can have a disparate impact on poor communities of color. A 2016 study reverse-engineered PredPol’s algorithm and found that it replicated “systemic bias” against over-policed communities of color and that historical crime data did not accurately predict future criminal activity. That sounds like more than "bad press".
- jrs95 9y agoIn my opinion it really isn’t. Looking at the studies it references, they’re not much different from “press” anyways. They read like the questions and data presented were selected to prove a political point. I’m not trying to just be ideological here either. I just don’t want the dominant ideology in academia and media to be used to attack and surpress technology that could ultimately benefit the communities they’re allegedly harming.
- Spooky23 9y ago"A man is known by the company he keeps" is a concept that phase been around for millenia, and is no longer valid today. The underlying question with all of these types of issues is that you are doing something "with a computer". Beat cops in community policing models do the same thing in an analog world. Good detectives know the territory, know the players and know where to look for things. At what point do do we want to stop? I don't know the answer.
- gowld 9y agoWhy is it no longer valid?
- QuercusMax 9y agoYeah - seems like it's no less valid. And quite possibly more valid.
- Jedi72 9y agoThe definition of "company one keeps" is open to interpretation. How many Facebook connections away do you think you are from a drug dealer? Take him away, boys!
- carlmr 9y agoSocial media also provides strength of connection. How often do you communicate with the drug dealer, at what times, how often is your GPS position in his house for 5 minutes. Probably everyone who isn't friendless on Facebook has a drug dealer in their immediate network. But using more data you can filter out the actual druggies with high confidence.
- arthur_pryor 9y ago> But using more data you can filter out the actual druggies with high confidence. i'm not comfortable with our justice system acting on the "high confidence" of some proprietary heuristic with a closed source implementation and little scientific evidence to support its claims, and with little ability for citizens to vet its workings at a detailed level, to see whether it implements their values or the values enshrined by the constitution. to say nothing of the fact that i have zero interest in using the criminal justice system to look for "druggies" (violent drug dealers, sure, because they're violent; drug abuse is a health problem that the criminal justice system is ill-equipped to deal with).
- throw7 9y agotoo bad carville didn't recommend palantir do some precrime analysis on politicians. but then that would be biting the hand that feeds you. surely palantir wouldn't stand for that.
- saas_co_de 9y agoYeah, maybe they should use their systems to analyze government contracting.
- deleted 9y ago[deleted]
- bilibala 9y agoI recommend reading the Math and Murder section of The Rise of Big Data Policing for a more researched view of how New Orleans is using Palantir (starts at 2nd half of page 40). https://books.google.com/books?id=-IPOAQAACAAJ&lpg=PA7&pg=PA40#v=onepage&q&f=false https://books.google.com/books?id=-IPOAQAACAAJ&lpg=PA7&pg=PA...
- aslewofmice 9y agoprivate data-driven policing will end up reinforcing any institutional racism under the guise of math
- MikkoFinell 9y agoThank you pc-bot for reminding us that math is racist.
- deleted 9y ago[deleted]
- s_m_t 9y agoAllowing environments where violent criminals and murderers are allowed to thrive IS institutional racism
- uoaei 9y agoAnd giving officers the ability to claim anyone fits within that tiny subset of the population based on spurious correlations is institutional racism. Protect the innocent, or sacrifice them for the good of catching some baddies?
- aaronbrethorst 9y ago99% Invisible had a great episode that touched on this last Fall. Well worth 20 minutes of your time: https://99percentinvisible.org/episode/the-age-of-the-algorithm/ https://99percentinvisible.org/episode/the-age-of-the-algori...
- flother 9y agoAnd Cathy O’Neil's book Weapons of Math Destruction, mentioned in the episode, is also well worth reading. https://www.amazon.co.uk/dp/0141985410 https://www.amazon.co.uk/dp/0141985410
- imaginenore 9y ago
- cinquemb 9y agoI look forward when systems like these start having to adapt to actors start using cointel tools and seed information into their systems with the intent to see how decisions are made from them to start gaming it for their own ends.
- deleted 9y ago[deleted]
- deleted 9y ago[deleted]
- tyfon 9y agoThe game Watch Dogs 2 is actually quite on target regarding this. In the game, the TPTB puts a non-scientific bias into predictive algorithms for certain neighbourhoods making the population of those areas pay more in insurance and be targeted by police a lot more often. It's a dangerous path to take as suddenly you can be labeled a criminal just for living in the wrong place or even worse if the police put their bias into the algorithms.
- galdosdi 9y agoHistory would be repeating itself. This was one of the factors in the burning down of much of the south Bronx in the 1970s. The city was broke and asked RAND Corporation to scientifically determine the least necessary firehouses to close. Their methods were in retrospect biased and just plain inaccurate, as firehouses and police precincts in poor neighborhoods were closed just as crime and fires were spiking, creating some sick sort of positive feedback loop. There are zip codes in the south Bronx that lost 90%+ of their housing. It's incredible. Beware of those who would use the word "science" to justify their preferences without explaining in detail.
- NutriSugar 9y agoThe systems also encode in existing discrimination. For example, women are less likely to be investigated for a crime, charged with a crime, convicted of a crime, and receive lower sentences when convicted. In some cases (especially involving sex crimes) there is explicit sex based bias written into the law itself. This means a lot of data on who commits crimes has a strong sexist bias, so any automated algorithms have a strong possibility of reinforcing the existing bias. The same will happen for race, class, and many other factors. There are also cases where society wants an incorrect bias put into place. For example, parole risk assessment software vastly underrates the threat of certain classes of criminals compared to what society and police think it should because there are major myths about rehabilitation and recidivism that are as popular as they are wrong. Perhaps the worst part is a total lack of transparency in the existing algorithms that determine risk. With enough data you can reverse engineer it, but that doesn't give the same impact as seeing the rules themselves. For example, one parole group I developed software for had a risk rating that appeared to automatically rate women a risk level lower for the same crime. Imagine if the same was done based on race.
- mtgx 9y agoMicrosoft has been doing some of this work for NYPD for some years, too.
- tyu100 9y agoFrankly, cities like New Orleans, Baltimore and New Orleans need systems like Palantir, drones, license plate readers and etc.. They are like something out of Hobbes right now, with no effective state authority and violence levels unseen elsewhere in the developed world. Once basic rule-of-law is established then programs like those can be scaled back but a lot of other countries would have established a state of emergency if they were confronted with the levels of violence seen in some of these American big cities. There's already a big push for body-worn camers for police, this is just a natural extension.
- Casseres 9y ago> programs like those can be scaled back "can" does not equal "will"
- justicezyx 9y agoTurns out China still copies from USA, even when it's for surveillance
- youdontknowtho 9y agoThe New Orleans police department is so insanely corrupt that if I related some of the stories I've heard from people who live there you would not believe them.
- fokinsean 9y agoCare to share one?
- youdontknowtho 9y agoA girl I dated had a NOPD cop that stalked her. He invited her to an "evidence party". She ended up leaving the city.
- ignoramceisblis 9y agoReminds me of one example of the dangers of "predictive policing", when controlled by one unchecked group: http://opentranscripts.org/transcript/tyranny-algorithms-predictive-policing-israel/ http://opentranscripts.org/transcript/tyranny-algorithms-pre... And https://www.theguardian.com/world/2014/sep/12/israeli-intelligence-unit-testimonies https://www.theguardian.com/world/2014/sep/12/israeli-intell... and https://www.theguardian.com/world/2014/sep/12/israeli-intelligence-veterans-letter-netanyahu-military-chiefs https://www.theguardian.com/world/2014/sep/12/israeli-intell....