5 ms·

Microsoft won’t sell police its facial-recognition technology

110 points by longdefeat 6y ago

Simulacra 6y ago
So..they'll just use it for themselves?
- AlexandrB 6y ago
  No, they're waiting "until there is a federal law regulating the technology". Then they can sell it with the PR cover of legislation. In the meantime they can still sell it to militaries, intelligence agencies, and police departments who are not governed by the US congress.
moksly 6y ago
I’m not American, but isn’t the police about the only people you should ever trust (if any) with facial-recognition technology? If you can’t trust your police with it, then there is something fundamentally wrong with your society.
- intopieces 6y ago
  The police have the power of the government behind them, which in the US literally has the ability to kill you in some states. At least if it's in the hands of corporations there's still a chance that the only entity more powerful (the government) will do something about it (through regulation). There is something fundamentally wrong with American society, though. No arguments there.
- jfengel 6y ago
  You got it in one: we don't trust our police with it, and there is something fundamentally wrong with society. That is why there have been calls to radically re-think, or even eliminate, policing in the US. This announcement coincides with protests against police brutality, at which many police have behaved brutally. That was sparked by an outright homicide by a police officer, captured on video, of a man who was subdued and presented no threat -- while other police officers watched, and many others have subsequently attempted to justify. The "something fundamentally wrong" is very complex and subject to genuine debate, but it's not subject to debate that whatever it is, people don't trust the police.
  - Lendal 6y ago
    It's less complex if you look at all these problems as symptoms. At the core is the racism. The gratuitous brutality, the lying, the planting of evidence, the us-versus-them attitude, the lawlessness, the replacement of the American flag with a Thin Blue Line flag, all of that could never have gotten so bad if racism wasn't at the core driving everything else.
    - phkahler 6y ago
      >> It's less complex if you look at all these problems as symptoms. At the core is the racism. I dont agree. Racism is a problem but I think corruption is the core. The police are often taught that they are above the law, starting with quotas - which encourage ticketing people for minor, subjective, or fictitious infractions. Moving up to things like parallel construction (lying in court), civil forfeiture (stealing from people), coercing confessions, and more. Then sprinkle in speeding when on duty when there is no reason, or illegally parking at McDonalds to run in for a burger. From small to significant infractions they give themselves and each other a pass, or are taught or encouraged to do wrong. In an environment like that you drop in a couple racists and well... There are studies on workplace safety. If you want to prevent fatal accidents you start by creating a culture of safety from the bottom. You clean up work spaces. You take care of trip hazards. You measure the number of bandages used each month and take measures to prevent those accidents. Over time this results in fewer fatalities even on large construction projects. I think the same applies to corruption.
      - AcerbicZero 6y ago
        I'm in agreement with you. While its obvious that race is a factor in at least a subset of incidents with the police (and those incidents should be investigated to the fullest), the vast majority of issues stemming from police interaction can effect anyone of any color. This is a legislative and judicial issue as well. Politicians, a fair number of whom are former judges, pass terrible legislation that is then prosecuted by DA's with little to no regard for justice (only conviction rates matter, right?), while cases are judged by former DA's who've managed to move up the chain a little. The management layer is a circle of incestuous self aggrandizing bullshit, and while cops are the boots stepping on the people, someone else to managing where, and how most of that stepping happens.
- deleted 6y ago
  [deleted]
- AlexandrB 6y ago
  > If you can’t trust your police with it, then there is something fundamentally wrong with your society. I disagree. Why should anyone at all be trusted with facial recognition technology? What if facial recognition technology is a fundamental enabler of dictatorship?
  - deleted 6y ago
    [deleted]
- dblohm7 6y ago
  The police should not have access to that kind of tech regardless of country, IMHO.
  - shultays 6y ago
    Why? Assuming we have a non-corrupt police force it is a great tool for finding criminals with warrants
    - deleted 6y ago
      [deleted]
    - GiorgioG 6y ago
      Such a police force only exists in fiction novels and on TV. It's not even just corruption that we have to be concerned about, it's just general incompetence - which exists in every facet of life. The problem is the police can arrest someone who has done nothing wrong and there are very few meaningful consequences. Once you are embroiled in the system, it is an expensive, slow process to get out of (if that's even possible.)
    - AnthonyMouse 6y ago
      > Assuming we have a non-corrupt police force it is a great tool for finding criminals with warrants It's actually kind of terrible at that, because it's not perfectly accurate. Even if they could make it 99% accurate, the country has 300+ million people in it and for every person you told it to match, it'd give you three million false positives. What it's used for is mass surveillance. You capture a face and there are millions of possible matches, but hey, one of the matches just bought something with a credit card 50 meters from there. Or they're carrying a cell phone billed to a person whose DMV photo matches that face. Then they put the cell phone down and travel on a vector from there past twelve more cameras. You get everyone's location history and meatspace social graph in a database. That's way more useful to an oppressive regime than any kind of legitimate use, because you can get the same information from a suspect under investigation by conducting surveillance of only them, e.g. put a GPS on their car. Retroactive surveillance can be used by future regimes to observe past behavior. Even if your existing government is trustworthy, there is an election every four years and turnkey fascism is inherently dangerous. Meanwhile retroactivity not all that useful for legitimate purposes because criminals who are actively committing new crimes can be caught by non-retroactive surveillance specifically of them, whereas there is a much weaker state interest in interest in catching criminals who have already reformed and stopped committing new crimes.
- GiorgioG 6y ago
  I'm generalizing here, but many Americans don't trust the government with surveilling the general populous. What if they use facial recognition to arrest you and that facial-recognition is flawed? How do you prove it? If a corporation uses facial recognition to count how many times I walked into a 7-11, I care much less about that than I do about the authorities building a database and watching every step people take in public. Yes maybe it's a little tinfoil hat to say, but the police/district attorneys are people and some of them are good, some bad. Why give them a tool that can be abused and misused against you?
- Nasrudith 6y ago
  The exact opposite, they are the only ones you should never trust because of other powers. To everybody else because they lack the enforcement powers they would mostly just be annoying with it. With police powers combined with a surveilance apparatus it becomes a horrifyingly abusable justification and tracking engine along with a wonderful way to insert pretexts for arresting anybody they don't like. Previously the search process would get slapped down along the way either from warrants or being judged a waste of resources. It is sort of a bleach and ammonia thing. One or the other is more or less fine with some potential for both legitimate use and abuse but together are a horribly toxic combination which is guaranteed to hurt and kill innocent people.
m0xte 6y ago
Perhaps if it has moral consequences they should delete the code?
- diegoperini 6y ago
  It can also have moral consequences if they deleted it.
  - m0xte 6y ago
    The answer with all trade-offs is which is worse. I suspect that one is worse than the other.
pilif 6y ago
But it happily sells GitHub enterprise to the ICE.
- deleted 6y ago
  [deleted]
thebouv 6y ago
But they'll sell it to a third party. Who can then sell it to the police. So this just means they won't directly sell it, but they'll still make their money.
- seph-reed 6y ago
  Nasty PR teamses. Trixies. Trixies!
- electricviolet 6y ago
  It would be easy for them to put a clause in their contracts with third parties that forbids them from reselling it to the police. Not necessarily saying Microsoft will do that, but they could do it if they're committed to this.
  - toast0 6y ago
    PR gain for now, and further push against First Sale doctrine for later, win win for Microsoft.
    - electricviolet 6y ago
      I don't know much in this area so I might be wrong, but wouldn't first sale doctrine not apply to software as a service?
jokoon 6y ago
At this point I guess ML developers should just stop working on anything related to face recognition...
- chickenpotpie 6y ago
  There’s still a lot of ethical uses of facial recognition. For example, finding missing persons.
beebmam 6y ago
Frankly, I think this is hypocritical. Satya Nadella has explicitly stated that Microsoft will not withhold technology from democratically elected governments. In this context, why is ethical for Microsoft to build the US military a "war planning and operations" cloud as part of the JEDI contract? In what reality is selling facial-recognition technology to police somehow less ethical than making the US military, in their own terms, "more lethal"? If Microsoft rejects the police for being human rights abusers, they should do the same for the US military, which has regularly violated human rights around the world.
- kasey_junk 6y ago
  I don’t necessarily believe this but it’s fairly easy to craft an argument for arming military forces separately from police forces. In a well functioning government the police would operate in completely different theaters & under completely different rules than the military. One can easily argue that many of the US’s current problems have to do with getting that blend incorrect but it doesn’t obviate the idea that done tech should go to one but not the other.
  - grawprog 6y ago
    >operate in completely different theaters & under completely different rules than the military. Killing people is really no different anywhere in the world. They're ok with their technology being used to kill, just so long as it's only people in certain place in the world and done by a certain group. What difference is it to me if I'm shot by a cop or a soldier? Either way I'm being shot by someone, likely far more well armed and armoured than myself, at the behest of the government. Who cares whether their uniform is blue or green?
    - kasey_junk 6y ago
      Again, this is an argument I’m fairly uncomfortable making but logically it is not hard to distinguish between an invading foreign army and it’s soldiery vs a bank robber. They demand different kinds of violence. The issue is that modern policing and invasion are not that black and white.
zests 6y ago
Doesn’t this just mean the police will buy from Palantir now? This problem needs a legislative solution.
nimbius 6y ago
I mean, i guess its a step but it misses the real opus of US policing in my opinion. AI and facial recognition require real investigative skill because they are still high risk technologies compared to current policing tactics. As it stands, cops mostly arrest brown and black low income people on capricious charges because the revenue is predictable and the suspects rarely have access to meaningful council that wont immediately direct them to plea-bargain. the revenue from this policing is nontrivial in the city budget, and it puts warm bodies in profitable corporate jails that many deficit-run cities are fully dependent upon to service their debt. AI and Facial recognition introduce the very real potential to arrest someone who not only isnt your suspect, but is powerful enough to fight back and make you look very bad. You cross over from comfy cop to officer who will need to be able to prove a crime based on evidence. for a lot of law enforcement that have become versed in their union bylaws, buying a product that can get them fired if they arent sharp gumshoes to solve real crime isnt an appealing idea anyway. And since we hire most cops from our boots-on-ground military troops, the argument could be made that most arent the sharpest sherlocks. Another problem is police overloading. cops are already social workers, dog catchers, school truancy enforcers, and crisis management mental health workers. Theres not much a cop in the US does not do if you call them. the idea of now making them a cloud-based AI guru at some level is likely going to be met with some resistance. What does the FR do? and what additional duties will cops have to absorb once its online? Is every face at a busy festival going to have cops chasing lookalikes? how do you get help if facial recognition is broken?

110 points by longdefeat 6y ago

Simulacra 6y ago

So..they'll just use it for themselves?

AlexandrB 6y ago
No, they're waiting "until there is a federal law regulating the technology". Then they can sell it with the PR cover of legislation. In the meantime they can still sell it to militaries, intelligence agencies, and police departments who are not governed by the US congress.

moksly 6y ago

I’m not American, but isn’t the police about the only people you should ever trust (if any) with facial-recognition technology? If you can’t trust your police with it, then there is something fundamentally wrong with your society.

intopieces 6y ago
The police have the power of the government behind them, which in the US literally has the ability to kill you in some states. At least if it's in the hands of corporations there's still a chance that the only entity more powerful (the government) will do something about it (through regulation). There is something fundamentally wrong with American society, though. No arguments there.
jfengel 6y ago
You got it in one: we don't trust our police with it, and there is something fundamentally wrong with society. That is why there have been calls to radically re-think, or even eliminate, policing in the US. This announcement coincides with protests against police brutality, at which many police have behaved brutally. That was sparked by an outright homicide by a police officer, captured on video, of a man who was subdued and presented no threat -- while other police officers watched, and many others have subsequently attempted to justify. The "something fundamentally wrong" is very complex and subject to genuine debate, but it's not subject to debate that whatever it is, people don't trust the police.
- Lendal 6y ago
  It's less complex if you look at all these problems as symptoms. At the core is the racism. The gratuitous brutality, the lying, the planting of evidence, the us-versus-them attitude, the lawlessness, the replacement of the American flag with a Thin Blue Line flag, all of that could never have gotten so bad if racism wasn't at the core driving everything else.
  - phkahler 6y ago
    >> It's less complex if you look at all these problems as symptoms. At the core is the racism. I dont agree. Racism is a problem but I think corruption is the core. The police are often taught that they are above the law, starting with quotas - which encourage ticketing people for minor, subjective, or fictitious infractions. Moving up to things like parallel construction (lying in court), civil forfeiture (stealing from people), coercing confessions, and more. Then sprinkle in speeding when on duty when there is no reason, or illegally parking at McDonalds to run in for a burger. From small to significant infractions they give themselves and each other a pass, or are taught or encouraged to do wrong. In an environment like that you drop in a couple racists and well... There are studies on workplace safety. If you want to prevent fatal accidents you start by creating a culture of safety from the bottom. You clean up work spaces. You take care of trip hazards. You measure the number of bandages used each month and take measures to prevent those accidents. Over time this results in fewer fatalities even on large construction projects. I think the same applies to corruption.
    - AcerbicZero 6y ago
      I'm in agreement with you. While its obvious that race is a factor in at least a subset of incidents with the police (and those incidents should be investigated to the fullest), the vast majority of issues stemming from police interaction can effect anyone of any color. This is a legislative and judicial issue as well. Politicians, a fair number of whom are former judges, pass terrible legislation that is then prosecuted by DA's with little to no regard for justice (only conviction rates matter, right?), while cases are judged by former DA's who've managed to move up the chain a little. The management layer is a circle of incestuous self aggrandizing bullshit, and while cops are the boots stepping on the people, someone else to managing where, and how most of that stepping happens.
deleted 6y ago
[deleted]
AlexandrB 6y ago
> If you can’t trust your police with it, then there is something fundamentally wrong with your society. I disagree. Why should anyone at all be trusted with facial recognition technology? What if facial recognition technology is a fundamental enabler of dictatorship?
- deleted 6y ago
  [deleted]
dblohm7 6y ago
The police should not have access to that kind of tech regardless of country, IMHO.
- shultays 6y ago
  Why? Assuming we have a non-corrupt police force it is a great tool for finding criminals with warrants
  - deleted 6y ago
    [deleted]
  - GiorgioG 6y ago
    Such a police force only exists in fiction novels and on TV. It's not even just corruption that we have to be concerned about, it's just general incompetence - which exists in every facet of life. The problem is the police can arrest someone who has done nothing wrong and there are very few meaningful consequences. Once you are embroiled in the system, it is an expensive, slow process to get out of (if that's even possible.)
  - AnthonyMouse 6y ago
    > Assuming we have a non-corrupt police force it is a great tool for finding criminals with warrants It's actually kind of terrible at that, because it's not perfectly accurate. Even if they could make it 99% accurate, the country has 300+ million people in it and for every person you told it to match, it'd give you three million false positives. What it's used for is mass surveillance. You capture a face and there are millions of possible matches, but hey, one of the matches just bought something with a credit card 50 meters from there. Or they're carrying a cell phone billed to a person whose DMV photo matches that face. Then they put the cell phone down and travel on a vector from there past twelve more cameras. You get everyone's location history and meatspace social graph in a database. That's way more useful to an oppressive regime than any kind of legitimate use, because you can get the same information from a suspect under investigation by conducting surveillance of only them, e.g. put a GPS on their car. Retroactive surveillance can be used by future regimes to observe past behavior. Even if your existing government is trustworthy, there is an election every four years and turnkey fascism is inherently dangerous. Meanwhile retroactivity not all that useful for legitimate purposes because criminals who are actively committing new crimes can be caught by non-retroactive surveillance specifically of them, whereas there is a much weaker state interest in interest in catching criminals who have already reformed and stopped committing new crimes.
GiorgioG 6y ago
I'm generalizing here, but many Americans don't trust the government with surveilling the general populous. What if they use facial recognition to arrest you and that facial-recognition is flawed? How do you prove it? If a corporation uses facial recognition to count how many times I walked into a 7-11, I care much less about that than I do about the authorities building a database and watching every step people take in public. Yes maybe it's a little tinfoil hat to say, but the police/district attorneys are people and some of them are good, some bad. Why give them a tool that can be abused and misused against you?
Nasrudith 6y ago
The exact opposite, they are the only ones you should never trust because of other powers. To everybody else because they lack the enforcement powers they would mostly just be annoying with it. With police powers combined with a surveilance apparatus it becomes a horrifyingly abusable justification and tracking engine along with a wonderful way to insert pretexts for arresting anybody they don't like. Previously the search process would get slapped down along the way either from warrants or being judged a waste of resources. It is sort of a bleach and ammonia thing. One or the other is more or less fine with some potential for both legitimate use and abuse but together are a horribly toxic combination which is guaranteed to hurt and kill innocent people.

m0xte 6y ago

Perhaps if it has moral consequences they should delete the code?

diegoperini 6y ago
It can also have moral consequences if they deleted it.
- m0xte 6y ago
  The answer with all trade-offs is which is worse. I suspect that one is worse than the other.

pilif 6y ago

But it happily sells GitHub enterprise to the ICE.

deleted 6y ago
[deleted]

thebouv 6y ago

But they'll sell it to a third party. Who can then sell it to the police. So this just means they won't directly sell it, but they'll still make their money.

seph-reed 6y ago
Nasty PR teamses. Trixies. Trixies!
electricviolet 6y ago
It would be easy for them to put a clause in their contracts with third parties that forbids them from reselling it to the police. Not necessarily saying Microsoft will do that, but they could do it if they're committed to this.
- toast0 6y ago
  PR gain for now, and further push against First Sale doctrine for later, win win for Microsoft.
  - electricviolet 6y ago
    I don't know much in this area so I might be wrong, but wouldn't first sale doctrine not apply to software as a service?

jokoon 6y ago

At this point I guess ML developers should just stop working on anything related to face recognition...

chickenpotpie 6y ago
There’s still a lot of ethical uses of facial recognition. For example, finding missing persons.

beebmam 6y ago

Frankly, I think this is hypocritical. Satya Nadella has explicitly stated that Microsoft will not withhold technology from democratically elected governments. In this context, why is ethical for Microsoft to build the US military a "war planning and operations" cloud as part of the JEDI contract? In what reality is selling facial-recognition technology to police somehow less ethical than making the US military, in their own terms, "more lethal"? If Microsoft rejects the police for being human rights abusers, they should do the same for the US military, which has regularly violated human rights around the world.

kasey_junk 6y ago
I don’t necessarily believe this but it’s fairly easy to craft an argument for arming military forces separately from police forces. In a well functioning government the police would operate in completely different theaters & under completely different rules than the military. One can easily argue that many of the US’s current problems have to do with getting that blend incorrect but it doesn’t obviate the idea that done tech should go to one but not the other.
- grawprog 6y ago
  >operate in completely different theaters & under completely different rules than the military. Killing people is really no different anywhere in the world. They're ok with their technology being used to kill, just so long as it's only people in certain place in the world and done by a certain group. What difference is it to me if I'm shot by a cop or a soldier? Either way I'm being shot by someone, likely far more well armed and armoured than myself, at the behest of the government. Who cares whether their uniform is blue or green?
  - kasey_junk 6y ago
    Again, this is an argument I’m fairly uncomfortable making but logically it is not hard to distinguish between an invading foreign army and it’s soldiery vs a bank robber. They demand different kinds of violence. The issue is that modern policing and invasion are not that black and white.

zests 6y ago

Doesn’t this just mean the police will buy from Palantir now? This problem needs a legislative solution.

nimbius 6y ago

I mean, i guess its a step but it misses the real opus of US policing in my opinion. AI and facial recognition require real investigative skill because they are still high risk technologies compared to current policing tactics. As it stands, cops mostly arrest brown and black low income people on capricious charges because the revenue is predictable and the suspects rarely have access to meaningful council that wont immediately direct them to plea-bargain. the revenue from this policing is nontrivial in the city budget, and it puts warm bodies in profitable corporate jails that many deficit-run cities are fully dependent upon to service their debt. AI and Facial recognition introduce the very real potential to arrest someone who not only isnt your suspect, but is powerful enough to fight back and make you look very bad. You cross over from comfy cop to officer who will need to be able to prove a crime based on evidence. for a lot of law enforcement that have become versed in their union bylaws, buying a product that can get them fired if they arent sharp gumshoes to solve real crime isnt an appealing idea anyway. And since we hire most cops from our boots-on-ground military troops, the argument could be made that most arent the sharpest sherlocks. Another problem is police overloading. cops are already social workers, dog catchers, school truancy enforcers, and crisis management mental health workers. Theres not much a cop in the US does not do if you call them. the idea of now making them a cloud-based AI guru at some level is likely going to be met with some resistance. What does the FR do? and what additional duties will cops have to absorb once its online? Is every face at a busy festival going to have cops chasing lookalikes? how do you get help if facial recognition is broken?