11 ms·

Microsoft bans U.S. police from using enterprise AI tool for facial recognition

254 points by coloneltcb 2y ago

ldng 2y ago
Wait, is that a decision a company is allowed to make ? Is it the beginning of cyberpunk ? Feels like they're just protecting their ass while keeping overselling their product. Not that law enforcement should use AI irresponsibly and without boundaries, but I'm not sure it is the responsibility of a company to make that kind of call.
- treyd 2y ago
  You generally can't force a party to enter into a contract with another party that they don't want to.
  - blackeyeblitzar 2y ago
    Utilities and carriers can’t just pick and choose customers totally. For example, your electrical utility cannot deny your service based on your political speech or profession or whatever. Even if they are privately owned. Personally, I think companies above a certain size should not be able to deny customers because they are effectively like public agencies or utility services due to their size. Banks, big tech, large social media platforms should be regulated like utilities in my opinion.
    - text0404 2y ago
      microsoft's enterprise AI tool isn't a utility though, and they're not banning police on political or moral grounds.
      - blackeyeblitzar 2y ago
        > microsoft's enterprise AI tool isn't a utility though It’s part of an infrastructure as a service platform, which is why I view it as utility. But I also am saying we need to broaden what we call a utility for the modern world. Not having access to large platforms (like social media) or centralized services (banking) or large companies (which have lots of market share and are one of a few options) is very damaging. What are your thoughts on that? > and they're not banning police on political or moral grounds. How do you know this? This seems like it would almost certainly be the result of pressure from political groups.
        pessimizer 2y ago
        > It’s part of an infrastructure as a service platform, which is why I view it as utility. They haven't securely broadened it to the internet or mobile phone service, and you want them to broaden it to individual privately-produced software packages. So the police can't be refused as a customer.
    - deleted 2y ago
      [deleted]
    - kube-system 2y ago
      Of all the things that could be considered a public utility, Azure OpenAI Service ranks somewhere between Cabbage Patch dolls and Crystal Pepsi. Not only can everyone live without it, almost everyone on the planet does live without it.
    - atonse 2y ago
      But utilities also have those rules because they are granted de-facto monopolies in their geographic areas. It's not like I can get electricity from another provider if there is literally only one in my area. And what they're providing (water, sewage, electricity) is deemed essential. Even with banks, if one bank rejects me, I can (hypothetically) go to another one down the street.
      - blackeyeblitzar 2y ago
        Companies can be damaging and anti competitive even if they aren’t total monopolies. Being an oligopoly or having a significant portion of market share is enough. Keep in mind many of these companies have more users than most national governments do - and they have soft power on a similar scale. Another angle is network effects - social media companies face reduced competition due to this and so they too should be regulated above a certain size since there isn’t enough competition or choice there, because the value of access to those particular platforms is very high due to their size. > Even with banks, if one bank rejects me, I can (hypothetically) go to another one down the street. What if they all reject you though? For example what if all banks decide they are better off debanking someone to stay in the good side of politicians who might punish them otherwise (like via regulation)? Some of these services are fundamental and should be forced to provide service no matter what.
    - singleshot_ 2y ago
      When it comes to wanting big companies to follow rules (a noble pursuit) many wish that common carrier status could be imposed on a business. But remember why common carrier status exists. Imagine I rob a stagecoach and lie low until the next day, when I catch a train to the next town over. Is the train engineer my accomplice in the robbery? Or my accessory after the fact? Is the train company? He and it might be, except that the train company “holds itself out” as a transporter of goods and services open to all. In exchange, the law grants the company that holds itself out as a carrier for all the freedom not to worry about whether carriage is a criminal act for the customer. But without holding one’s self out as a carrier for all, there is no common carrier status. You cannot hold out a company as sufficiently large; they must hold themselves out as wanting this status. When I see people clamor for social media platforms, I get doubly confused, because not only have these companies never held themselves out as common carriers, they would take nothing they do not already have (under 230) from doing so.
      - everforward 2y ago
        > Is the train engineer my accomplice in the robbery? Or my accessory after the fact? Is the train company? No, even without common carrier status. Being an accessory to a crime is much more narrow than "unknowingly conveyed a criminal during routine operation of a transportation company". That broad of an interpretation makes everything being an accessory to the crime. The federal government created and backs the bills the thief stole, so they helped. The city built the roads they used, so they help. Old Mister McCollem down the lane stopped walking so the thief didn't run them over, which might have helped. > But without holding one’s self out as a carrier for all, there is no common carrier status. You cannot hold out a company as sufficiently large; they must hold themselves out as wanting this status. This is not true, you're misconstruing the difference between companies that had mens rea in transporting illegal goods and companies that don't. Common carrier is an involuntary status. The government decides whether a company is or is not a common carrier and is obligated to provide services to anyone that asks (basically). Common carriers are almost categorically excluded from liability for the goods they transport because they are effectively incapable of mens rea. They did not choose to transport those goods, therefore they could not have had criminal intent in making a choice to provide service. Companies that are not common carriers _do_ have a choice when deciding whether to provide service, and thus are capable of having criminal intent when making that decision. Inversely, companies that choose to act like a common carrier _do not_ receive the same protections as an actual common carrier. Their decision to provide service to everyone without asking questions is still a decision, and can meet the mens rea bar. The most well known cases I can think of here would be The Pirate Bay and Silk Roads. Both behave as if they were common carriers and will sell/convey almost whatever you want without questions. They do not receive immunity for not asking questions, because the intent of that policy is to allow (encourage, really) illegal usage. On the flip side, the Post Office/UPS have indemnity for conveying drugs bought on Silk Roads because they actually are common carriers. They didn't decide to just not check packages, they are legally prohibited from invasive package checking and denying service to customers they find sketchy (would need a fact check on the prohibition of checking packages; I believe there are limits on what they can check, but I don't know offhand what they are).
    - whaleofatw2022 2y ago
      I think in this case, a bigger factor would be this is a 'we will not attest to accuracy in court'. As an odd sort of comparison, this is why you can't just take a magic 8 ball into the courtroom and shake it; 'for amusement only'
- miah_ 2y ago
  Corporations can make any judgement call about who can or should use their product. Though it does seem rare in the US for companies to prevent others from using their product based on ethical or moral reasons, but more should.
  - busterarm 2y ago
    US banks don't do business with the porn industry pretty famously.
    - Alupis 2y ago
      That is not due to morality though - that is due to the insane level of chargebacks and fraud revolving around payments within the porn industry.
      - HeatrayEnjoyer 2y ago
        Except it is. High shrinkage can be baked into the business model and digital media is a high margin business that can tolerate it. Other industries with high fraud rates don't get cut out wholesale.
        favorited 2y ago
        Exactly. Mastercard, Visa & Discover were perfectly happy to process PornHub payments for years - until enough external political pressure was applied. Religious special interest lobbying groups like like Focus on the Family and NCOSE (formerly known as Morality in Media) were celebrating when they stopped, as were Republican politicians like Josh Hawley (https://twitter.com/HawleyMO/status/1337111830976753672 https://twitter.com/HawleyMO/status/1337111830976753672).
        Alupis 2y ago
        > until enough external political pressure was applied This doesn't pass the sniff test. You have been able to purchase porn for decades in various mediums - why would a website suddenly invoke the ire of the "morality police"? Even more-so when porn is freely available in almost every case when it comes to the internet. Really though, it has nothing to do with morality - it is straight up the risk profile of those websites. There's even a movie about the start of the online paid-porn industry and all the stuff they had to do to figure out how not to bleed money on chargebacks etc.
- simonw 2y ago
  Forcing companies to sell their surveillance-enabling product to the police seems pretty cyberpunk-dystopian to me.
- duxup 2y ago
  Presumably if I have a service that I think is inappropriate for a use (for whatever reason, but let's say false positives) I can in fact limit what kind of use cases area allowed. I could argue that my tool used wrong presents a legal risk to ME because I let law enforcement do some terrible things that I know don't work. Semi-related, most cloud services do not allow crypto mining, the reason there being it's not cost effective. So whatever mining does happen is stolen credentials, or some inside company job where dude is using company resources for personal gain ... pretty much all illegal activity. So they ban it and I believe actively try to detect / aggressively disable it.
  - RajT88 2y ago
    I think what it's about, is they can squint a little bit and see a near future where there is massive class-action suits against Microsoft for all the harmful uses which inevitably flow from a police department uncritically using hallucinating AI for their day to day work.
- SpaceManNabs 2y ago
  > Wait, is that a decision a company is allowed to make ? Is it the beginning of cyberpunk ? Companies can choose their clients in a non cyperbunk world too.
- cthalupa 2y ago
  > Wait, is that a decision a company is allowed to make ? Is it the beginning of cyberpunk ? Yes. Freedom of association is literally part of the first amendment. The only limits are around protected classes. How in the world is forcing a company to do business with someone a LESS dystopian solution than allowing them to decide who they do business with?
megraf 2y ago
Of course they did. They have to pay separate licensing for the use of DAS – which is developed by M$ and resold from NYPD. https://en.wikipedia.org/wiki/Domain_Awareness_System https://en.wikipedia.org/wiki/Domain_Awareness_System
- beeboobaa3 2y ago
  > 9,000 CCTV cameras, owned either by the NYPD or private actors I wonder what kind of person volunteers their camera for the surveillance apparatus.
  - themoonisachees 2y ago
    Businesses point cameras at the public for the sole purpose of surveillance. I'm not talking other businesses, I'm talking business built to sell and launder surveillance footage.
  - mikestew 2y ago
    The business owner who might have had their fill of break-ins, people pooping in front of their store, et al., for one. I'm sure "private actors" is not all just Ring doorbells.
- iceyest 2y ago
  Wow, I had no idea it was this bad. I am not really suprised the lengths American spying goes though. Glad not to be living in New York. >The Domain Awareness System is the largest digital surveillance system in the world I wonder how it compares to China and if facial recognition tech is as pervasive in America as it is in China.
  - pfortuny 2y ago
    There’s London too, totally dystopian.
    - vidarh 2y ago
      Most cameras in the UK are private, rarely working well, and usually not networked or easily accessible to the police. It could be better, but the idea of the UK as a surveillance state is seriously overblown outside of maybe a handful of streets.
MeImCounting 2y ago
This seems like a good thing. I would not trust US police to hold themselves to literally any standard of ethics as far as automation of this sort is concerned. I can easily see departments ignoring the outputs that dont fit into their agenda while running running with the outputs they do like. I would like to see more companies restricting what contracts theyll take based on ethics.
- ben_jones 2y ago
  AI is a giant liability shield. You can blatantly steal shit and profit of it. Of course enforcement agencies are going to have a field day with it. Announcements like this are just a pretense that they aren’t already counting the money.
- devmor 2y ago
  Unfortunately that’s not what’s happening here. The police still have access to this information and tooling. They just have a different license agreement and access point that allows Microsoft to extract more tax dollars.
  - Suppafly 2y ago
    There is probably an actual reason for it other than corporate greed. I'm guessing they have to silo the data separately on their cloud and that costs more. It's the same reason healthcare applications often cost more to host, they can't intermingle data in cloud instances and have to insure that it's only hosted in the US and what not. I imagine having to deal with legal procedures ensures a lot of additional time/money compared to other customers.
    - devmor 2y ago
      Oh I would certainly bet there's a lot of reasons it costs more money too. I'm just being reductivist and pessimist about it!
alwaysrunning 2y ago
How exactly are they going to stop the NYPD from using this service? Just not register it in their subscription as a service? And does it really matter? If I can use Azure's cognitive search and host my own model that doesn't stop me from using Azure for the same purpose.
- HeatrayEnjoyer 2y ago
  Using it unofficially through unauthorized workarounds brings doubt into the chain of custody and investigation integrity.
  - _DeadFred_ 2y ago
    Oh no, they will have to use parallel construction, something they TOTALLY don't have experience doing.
  - alwaysrunning 2y ago
    I'm not sure the courts would consider it a workaround, it's a legit way to host your own model. https://learn.microsoft.com/en-us/azure/machine-learning/how-to-deploy-models-llama?view=azureml-api-2&tabs=llama-three https://learn.microsoft.com/en-us/azure/machine-learning/how...
  - riffic 2y ago
    when cops and DAs use parallel construction to build a case, "integrity" is a mere technicality. https://en.wikipedia.org/wiki/Parallel_construction https://en.wikipedia.org/wiki/Parallel_construction
  - Maxious 2y ago
    Clearview AI created an unregulated facial recognition system and let individual cops use it for free as a "trial" Their superiors claim they had no idea https://www.abc.net.au/news/2020-07-13/afp-use-of-facial-recocognition-sofware-clearview-ai-revealed/12451554 https://www.abc.net.au/news/2020-07-13/afp-use-of-facial-rec...
- karaterobot 2y ago
option 2y ago
[flagged]
chefandy 2y ago
"In January, reporting by Bloomberg revealed that OpenAI is working with the Pentagon on a number of projects including cybersecurity capabilities — a departure from the startup’s earlier ban on providing its AI to militaries. Elsewhere, Microsoft has pitched using OpenAI’s image generation tool, DALL-E, to help the Department of Defense (DoD) build software to execute military operations, per The Intercept." Hm... to my eye, this just seems to be a great spin on essentially pushing cops or their vendors to sign (more expensive, I imagine) independent service contracts under the guise of oversight. Their principles seem to be as genuine as they were in their plea to the US government to cede control of LLMs, generally, to the big corporate players "for safety."
- SpaceManNabs 2y ago
  OpenAI has blatantly ignored most of its principles (which Elon called out surprisingly well). Insane to me that people ignore that.
  - chefandy 2y ago
    Seriously. > Insane to me that people ignore that. Yeah for sure, but I don't think this is a super representative crowd, though. People uncritically frolicking through all of these developments with AI-enhanced rose-colored-glasses glued to their faces are over-represented in the SV universe. I know a lot of non-tech-focused folks that are casting a very wary eye towards our impending incredible AI-powered corporate utopia. The only similarly optimistic people I know outside of tech are the devastatingly credulous fellows still trying to figure out how to recoup money on all of the NFTs they bought, and they seem to be much better at generating headlines than societal changes. But maybe I'm the one wearing the rose-colored glasses, here.
resource_waste 2y ago
Is this just PR stuff? Obviously enterprise tech companies (Google/MS) are going to be working with States. These State have to enforce the social contract so you are going to have blood on your hands. I don't really have skin in the game, I'm at the whim of these parties.
- warkdarrior 2y ago
  This is a great opportunity for open-source models to get a foothold in this lucrative market.
- nicklecompte 2y ago
  I suspect it's legal. The focus seems to be on face and speech recognition and I am sure MSFT's internal metrics on the AI's ability to recognize non-white faces/accents/etc are very bad. Even with LLMs, the existence of lawyers unknowingly using ChatGPT confabulations in court suggests that police officers will do the same with investigations or surveillance. A cop who takes Sam Altman's dumb tweets too seriously might think Azure's GPT-4 service can look at an evidence report and apply its AI Magic. Probably not something Microsoft wants to defend in court.
hm-nah 2y ago
IMHO, speech-to-text to LLMs is borderline spycraft already. “Are you wearing a wire?” The answer doesn’t mean merely recording the conversation. Now it could mean, a personalized “analyst” determining sentiment, correlating events/evidence/facts/(hallucinations)/building a prosecutors case…in real time!
advisedwang 2y ago
This is excellent. There will be blood on our hands if we allow our software to be used to more closely surveil and police people. Not only because the software as it currently exists is flaky, but because of how it will enviably be used. Certain communities will have it deployed against them more, parallel construction will hide when technology is used in violation of rights, police and juries will believe when technology has a "hit" that it proves guilt and not understand the caveats, etc etc. Some Microsoft employees have been calling for such limits for years [1, 2] [1] https://www.geekwire.com/2019/microsoft-github-workers-protest-ice-contracts-latest-demonstration-employee-activism/ https://www.geekwire.com/2019/microsoft-github-workers-prote... [2] https://www.cbsnews.com/news/microsoft-workers-asking-drop-police-contracts/ https://www.cbsnews.com/news/microsoft-workers-asking-drop-p...
- chefandy 2y ago
  If it's like what they did with the military, they're just trying to push them into a more expensive licensing scheme.
- rockinghigh 2y ago
  The NYPD has had a system built by Microsoft for 15 years: > Beginning in 2009, the NYPD, in partnership with Microsoft, built a powerful counterterrorism and policing tool called the Domain Awareness System (DAS). The DAS is a central platform used to aggregate data from internal and external closed-circuit television cameras (CCTV), license plate readers (LPRs), and environmental sensors, as well as 911 calls and other NYPD databases. The DAS uses an interactive dashboard interface to display real-time alerts whenever a 911 call is received or a sensor is triggered. The DAS also in- cludes mapping features that make it possible to survey and track targets. The DAS was originally built to support the Lower Manhattan Security Initiative (LMSI) – a public-private partnership – but has since expanded to cover the entire city, giving NYPD personnel direct access to thousands of cameras owned and operated by private orga- nizations. Until the development of the mobility platform and the mobile DAS system described below, the full capabilities of DAS have only been available to the Counterterrorism Bureau and a few other specially trained officers on desktop computers. As the NYPD upgrades its network, access to all DAS capabilities and resources will be expanded to all NYPD’s commands. https://www.nyc.gov/html/nypd/html/home/POA/pdf/Technology.pdf https://www.nyc.gov/html/nypd/html/home/POA/pdf/Technology.p...
deleted 2y ago
[deleted]
edot 2y ago
They are not banning them from using their AI tool. They're banning them from the currently-offered licensing structure. This is a profit-grab, not an ethical restriction. It is of the same vein as Sam Altman going to congress and spooking them in hopes of regulating AI so that they are the only ones allowed to build it. I wouldn't be shocked if we also saw a similar move to ban hospitals, schools, etc. from using the standard enterprise license under the guise of "we need to provide you with oversight for your vulnerable populations", and then charging them a hefty surcharge on top of the standard enterprise rate. "Responsible" AI is more profitable than standard AI - regardless of the nebulous nature of what "responsible" means.
- jMyles 2y ago
  Spot on. I'll explore whether this nit is worth picking: > "Responsible" AI is more profitable than standard AI - regardless of the nebulous nature of what "responsible" means. /s/regardless of/in part owing to
spxneo 2y ago
American law enforcement openly disregard the foundations from which the country was built upon yet pushes other countries to adopt ideas which it abandoned but won't admit it out of humiliation--that which we call "democracy" comes with the very hooks and strings it tells other countries to abandon. Just take a look at the countries that credulously adopted "democracy". They are not in a good shape and will probably never find itself out of the moving goal post that Western countries like to place on "lesser" countries. America is addicted to these 3 things: Oil, authoritarian labour, drugs. It's law enforcement is not meant to police those addictions at all but rather to ensure its continued uninterrupted supply even if it means violating its own principles that made the country great (about 40+ years ago). It's no wonder many Americans are yearning for past glory.
- lupire 2y ago
  Yes, the past glory of Legally mandated racial discrimination, and a war on Black people and Hispanics on a pretext of a War on Drugs.
deusum 2y ago
No one has pointed out that the AI companies are being sued for 'stealing' vast troves of copyrighted data, and the case is still in progress. If the police departments are complicit, then does copyright stand to lose de facto as AI will only be increasingly implemented while the case slogs through the courts?
- kcb 2y ago
  One bare minimum way to not get your data "stolen" is to not personally make it freely available on the internet.
flenserboy 2y ago
It is dangerous to allow groups of any sort to be excluded from licensing. This sort of behavior is far too easily weaponized against minority political, opinion, & affinity groups. While I'm not keen on the police having these tools in their hands, who else will be next? Who will be squeezed out of the economy because everyone who provides a particular sort of software or service decides to enact the same policies? This is why software should be sold, not licensed — a word processor should be treated no differently from a loaf of bread.
- tehwebguy 2y ago
  "First they came for the Police state and I said nothing..." lmao
- deleted 2y ago
  [deleted]
devindotcom 2y ago
FYI, Microsoft now says (we contacted before publishing) the policy we cited "contained an error" and was intended to only apply to facial recognition, and has been changed in the official code of conduct. We've updated the headline and body to reflect this change. A mod might want to change the HN hed too.
forgetfreeman 2y ago
[flagged]

254 points by coloneltcb 2y ago

ldng 2y ago

Wait, is that a decision a company is allowed to make ? Is it the beginning of cyberpunk ? Feels like they're just protecting their ass while keeping overselling their product. Not that law enforcement should use AI irresponsibly and without boundaries, but I'm not sure it is the responsibility of a company to make that kind of call.

treyd 2y ago
You generally can't force a party to enter into a contract with another party that they don't want to.
- blackeyeblitzar 2y ago
  Utilities and carriers can’t just pick and choose customers totally. For example, your electrical utility cannot deny your service based on your political speech or profession or whatever. Even if they are privately owned. Personally, I think companies above a certain size should not be able to deny customers because they are effectively like public agencies or utility services due to their size. Banks, big tech, large social media platforms should be regulated like utilities in my opinion.
  - text0404 2y ago
    microsoft's enterprise AI tool isn't a utility though, and they're not banning police on political or moral grounds.
    - blackeyeblitzar 2y ago
      > microsoft's enterprise AI tool isn't a utility though It’s part of an infrastructure as a service platform, which is why I view it as utility. But I also am saying we need to broaden what we call a utility for the modern world. Not having access to large platforms (like social media) or centralized services (banking) or large companies (which have lots of market share and are one of a few options) is very damaging. What are your thoughts on that? > and they're not banning police on political or moral grounds. How do you know this? This seems like it would almost certainly be the result of pressure from political groups.
      - pessimizer 2y ago
        > It’s part of an infrastructure as a service platform, which is why I view it as utility. They haven't securely broadened it to the internet or mobile phone service, and you want them to broaden it to individual privately-produced software packages. So the police can't be refused as a customer.
  - deleted 2y ago
    [deleted]
  - kube-system 2y ago
    Of all the things that could be considered a public utility, Azure OpenAI Service ranks somewhere between Cabbage Patch dolls and Crystal Pepsi. Not only can everyone live without it, almost everyone on the planet does live without it.
  - atonse 2y ago
    But utilities also have those rules because they are granted de-facto monopolies in their geographic areas. It's not like I can get electricity from another provider if there is literally only one in my area. And what they're providing (water, sewage, electricity) is deemed essential. Even with banks, if one bank rejects me, I can (hypothetically) go to another one down the street.
    - blackeyeblitzar 2y ago
      Companies can be damaging and anti competitive even if they aren’t total monopolies. Being an oligopoly or having a significant portion of market share is enough. Keep in mind many of these companies have more users than most national governments do - and they have soft power on a similar scale. Another angle is network effects - social media companies face reduced competition due to this and so they too should be regulated above a certain size since there isn’t enough competition or choice there, because the value of access to those particular platforms is very high due to their size. > Even with banks, if one bank rejects me, I can (hypothetically) go to another one down the street. What if they all reject you though? For example what if all banks decide they are better off debanking someone to stay in the good side of politicians who might punish them otherwise (like via regulation)? Some of these services are fundamental and should be forced to provide service no matter what.
  - singleshot_ 2y ago
    When it comes to wanting big companies to follow rules (a noble pursuit) many wish that common carrier status could be imposed on a business. But remember why common carrier status exists. Imagine I rob a stagecoach and lie low until the next day, when I catch a train to the next town over. Is the train engineer my accomplice in the robbery? Or my accessory after the fact? Is the train company? He and it might be, except that the train company “holds itself out” as a transporter of goods and services open to all. In exchange, the law grants the company that holds itself out as a carrier for all the freedom not to worry about whether carriage is a criminal act for the customer. But without holding one’s self out as a carrier for all, there is no common carrier status. You cannot hold out a company as sufficiently large; they must hold themselves out as wanting this status. When I see people clamor for social media platforms, I get doubly confused, because not only have these companies never held themselves out as common carriers, they would take nothing they do not already have (under 230) from doing so.
    - everforward 2y ago
      > Is the train engineer my accomplice in the robbery? Or my accessory after the fact? Is the train company? No, even without common carrier status. Being an accessory to a crime is much more narrow than "unknowingly conveyed a criminal during routine operation of a transportation company". That broad of an interpretation makes everything being an accessory to the crime. The federal government created and backs the bills the thief stole, so they helped. The city built the roads they used, so they help. Old Mister McCollem down the lane stopped walking so the thief didn't run them over, which might have helped. > But without holding one’s self out as a carrier for all, there is no common carrier status. You cannot hold out a company as sufficiently large; they must hold themselves out as wanting this status. This is not true, you're misconstruing the difference between companies that had mens rea in transporting illegal goods and companies that don't. Common carrier is an involuntary status. The government decides whether a company is or is not a common carrier and is obligated to provide services to anyone that asks (basically). Common carriers are almost categorically excluded from liability for the goods they transport because they are effectively incapable of mens rea. They did not choose to transport those goods, therefore they could not have had criminal intent in making a choice to provide service. Companies that are not common carriers _do_ have a choice when deciding whether to provide service, and thus are capable of having criminal intent when making that decision. Inversely, companies that choose to act like a common carrier _do not_ receive the same protections as an actual common carrier. Their decision to provide service to everyone without asking questions is still a decision, and can meet the mens rea bar. The most well known cases I can think of here would be The Pirate Bay and Silk Roads. Both behave as if they were common carriers and will sell/convey almost whatever you want without questions. They do not receive immunity for not asking questions, because the intent of that policy is to allow (encourage, really) illegal usage. On the flip side, the Post Office/UPS have indemnity for conveying drugs bought on Silk Roads because they actually are common carriers. They didn't decide to just not check packages, they are legally prohibited from invasive package checking and denying service to customers they find sketchy (would need a fact check on the prohibition of checking packages; I believe there are limits on what they can check, but I don't know offhand what they are).
  - whaleofatw2022 2y ago
    I think in this case, a bigger factor would be this is a 'we will not attest to accuracy in court'. As an odd sort of comparison, this is why you can't just take a magic 8 ball into the courtroom and shake it; 'for amusement only'
miah_ 2y ago
Corporations can make any judgement call about who can or should use their product. Though it does seem rare in the US for companies to prevent others from using their product based on ethical or moral reasons, but more should.
- busterarm 2y ago
  US banks don't do business with the porn industry pretty famously.
  - Alupis 2y ago
    That is not due to morality though - that is due to the insane level of chargebacks and fraud revolving around payments within the porn industry.
    - HeatrayEnjoyer 2y ago
      Except it is. High shrinkage can be baked into the business model and digital media is a high margin business that can tolerate it. Other industries with high fraud rates don't get cut out wholesale.
      - favorited 2y ago
        Exactly. Mastercard, Visa & Discover were perfectly happy to process PornHub payments for years - until enough external political pressure was applied. Religious special interest lobbying groups like like Focus on the Family and NCOSE (formerly known as Morality in Media) were celebrating when they stopped, as were Republican politicians like Josh Hawley (https://twitter.com/HawleyMO/status/1337111830976753672 https://twitter.com/HawleyMO/status/1337111830976753672).
        Alupis 2y ago
        > until enough external political pressure was applied This doesn't pass the sniff test. You have been able to purchase porn for decades in various mediums - why would a website suddenly invoke the ire of the "morality police"? Even more-so when porn is freely available in almost every case when it comes to the internet. Really though, it has nothing to do with morality - it is straight up the risk profile of those websites. There's even a movie about the start of the online paid-porn industry and all the stuff they had to do to figure out how not to bleed money on chargebacks etc.
simonw 2y ago
Forcing companies to sell their surveillance-enabling product to the police seems pretty cyberpunk-dystopian to me.
duxup 2y ago
Presumably if I have a service that I think is inappropriate for a use (for whatever reason, but let's say false positives) I can in fact limit what kind of use cases area allowed. I could argue that my tool used wrong presents a legal risk to ME because I let law enforcement do some terrible things that I know don't work. Semi-related, most cloud services do not allow crypto mining, the reason there being it's not cost effective. So whatever mining does happen is stolen credentials, or some inside company job where dude is using company resources for personal gain ... pretty much all illegal activity. So they ban it and I believe actively try to detect / aggressively disable it.
- RajT88 2y ago
  I think what it's about, is they can squint a little bit and see a near future where there is massive class-action suits against Microsoft for all the harmful uses which inevitably flow from a police department uncritically using hallucinating AI for their day to day work.
SpaceManNabs 2y ago
> Wait, is that a decision a company is allowed to make ? Is it the beginning of cyberpunk ? Companies can choose their clients in a non cyperbunk world too.
cthalupa 2y ago
> Wait, is that a decision a company is allowed to make ? Is it the beginning of cyberpunk ? Yes. Freedom of association is literally part of the first amendment. The only limits are around protected classes. How in the world is forcing a company to do business with someone a LESS dystopian solution than allowing them to decide who they do business with?

megraf 2y ago

Of course they did. They have to pay separate licensing for the use of DAS – which is developed by M$ and resold from NYPD. https://en.wikipedia.org/wiki/Domain_Awareness_System https://en.wikipedia.org/wiki/Domain_Awareness_System

beeboobaa3 2y ago
> 9,000 CCTV cameras, owned either by the NYPD or private actors I wonder what kind of person volunteers their camera for the surveillance apparatus.
- themoonisachees 2y ago
  Businesses point cameras at the public for the sole purpose of surveillance. I'm not talking other businesses, I'm talking business built to sell and launder surveillance footage.
- mikestew 2y ago
  The business owner who might have had their fill of break-ins, people pooping in front of their store, et al., for one. I'm sure "private actors" is not all just Ring doorbells.
iceyest 2y ago
Wow, I had no idea it was this bad. I am not really suprised the lengths American spying goes though. Glad not to be living in New York. >The Domain Awareness System is the largest digital surveillance system in the world I wonder how it compares to China and if facial recognition tech is as pervasive in America as it is in China.
- pfortuny 2y ago
  There’s London too, totally dystopian.
  - vidarh 2y ago
    Most cameras in the UK are private, rarely working well, and usually not networked or easily accessible to the police. It could be better, but the idea of the UK as a surveillance state is seriously overblown outside of maybe a handful of streets.

MeImCounting 2y ago

This seems like a good thing. I would not trust US police to hold themselves to literally any standard of ethics as far as automation of this sort is concerned. I can easily see departments ignoring the outputs that dont fit into their agenda while running running with the outputs they do like. I would like to see more companies restricting what contracts theyll take based on ethics.

ben_jones 2y ago
AI is a giant liability shield. You can blatantly steal shit and profit of it. Of course enforcement agencies are going to have a field day with it. Announcements like this are just a pretense that they aren’t already counting the money.
devmor 2y ago
Unfortunately that’s not what’s happening here. The police still have access to this information and tooling. They just have a different license agreement and access point that allows Microsoft to extract more tax dollars.
- Suppafly 2y ago
  There is probably an actual reason for it other than corporate greed. I'm guessing they have to silo the data separately on their cloud and that costs more. It's the same reason healthcare applications often cost more to host, they can't intermingle data in cloud instances and have to insure that it's only hosted in the US and what not. I imagine having to deal with legal procedures ensures a lot of additional time/money compared to other customers.
  - devmor 2y ago
    Oh I would certainly bet there's a lot of reasons it costs more money too. I'm just being reductivist and pessimist about it!

alwaysrunning 2y ago

How exactly are they going to stop the NYPD from using this service? Just not register it in their subscription as a service? And does it really matter? If I can use Azure's cognitive search and host my own model that doesn't stop me from using Azure for the same purpose.

HeatrayEnjoyer 2y ago
Using it unofficially through unauthorized workarounds brings doubt into the chain of custody and investigation integrity.
- _DeadFred_ 2y ago
  Oh no, they will have to use parallel construction, something they TOTALLY don't have experience doing.
- alwaysrunning 2y ago
  I'm not sure the courts would consider it a workaround, it's a legit way to host your own model. https://learn.microsoft.com/en-us/azure/machine-learning/how-to-deploy-models-llama?view=azureml-api-2&tabs=llama-three https://learn.microsoft.com/en-us/azure/machine-learning/how...
- riffic 2y ago
  when cops and DAs use parallel construction to build a case, "integrity" is a mere technicality. https://en.wikipedia.org/wiki/Parallel_construction https://en.wikipedia.org/wiki/Parallel_construction
- Maxious 2y ago
  Clearview AI created an unregulated facial recognition system and let individual cops use it for free as a "trial" Their superiors claim they had no idea https://www.abc.net.au/news/2020-07-13/afp-use-of-facial-recocognition-sofware-clearview-ai-revealed/12451554 https://www.abc.net.au/news/2020-07-13/afp-use-of-facial-rec...
karaterobot 2y ago

option 2y ago

[flagged]

chefandy 2y ago

"In January, reporting by Bloomberg revealed that OpenAI is working with the Pentagon on a number of projects including cybersecurity capabilities — a departure from the startup’s earlier ban on providing its AI to militaries. Elsewhere, Microsoft has pitched using OpenAI’s image generation tool, DALL-E, to help the Department of Defense (DoD) build software to execute military operations, per The Intercept." Hm... to my eye, this just seems to be a great spin on essentially pushing cops or their vendors to sign (more expensive, I imagine) independent service contracts under the guise of oversight. Their principles seem to be as genuine as they were in their plea to the US government to cede control of LLMs, generally, to the big corporate players "for safety."

SpaceManNabs 2y ago
OpenAI has blatantly ignored most of its principles (which Elon called out surprisingly well). Insane to me that people ignore that.
- chefandy 2y ago
  Seriously. > Insane to me that people ignore that. Yeah for sure, but I don't think this is a super representative crowd, though. People uncritically frolicking through all of these developments with AI-enhanced rose-colored-glasses glued to their faces are over-represented in the SV universe. I know a lot of non-tech-focused folks that are casting a very wary eye towards our impending incredible AI-powered corporate utopia. The only similarly optimistic people I know outside of tech are the devastatingly credulous fellows still trying to figure out how to recoup money on all of the NFTs they bought, and they seem to be much better at generating headlines than societal changes. But maybe I'm the one wearing the rose-colored glasses, here.

resource_waste 2y ago

Is this just PR stuff? Obviously enterprise tech companies (Google/MS) are going to be working with States. These State have to enforce the social contract so you are going to have blood on your hands. I don't really have skin in the game, I'm at the whim of these parties.

warkdarrior 2y ago
This is a great opportunity for open-source models to get a foothold in this lucrative market.
nicklecompte 2y ago
I suspect it's legal. The focus seems to be on face and speech recognition and I am sure MSFT's internal metrics on the AI's ability to recognize non-white faces/accents/etc are very bad. Even with LLMs, the existence of lawyers unknowingly using ChatGPT confabulations in court suggests that police officers will do the same with investigations or surveillance. A cop who takes Sam Altman's dumb tweets too seriously might think Azure's GPT-4 service can look at an evidence report and apply its AI Magic. Probably not something Microsoft wants to defend in court.

hm-nah 2y ago

IMHO, speech-to-text to LLMs is borderline spycraft already. “Are you wearing a wire?” The answer doesn’t mean merely recording the conversation. Now it could mean, a personalized “analyst” determining sentiment, correlating events/evidence/facts/(hallucinations)/building a prosecutors case…in real time!

advisedwang 2y ago

This is excellent. There will be blood on our hands if we allow our software to be used to more closely surveil and police people. Not only because the software as it currently exists is flaky, but because of how it will enviably be used. Certain communities will have it deployed against them more, parallel construction will hide when technology is used in violation of rights, police and juries will believe when technology has a "hit" that it proves guilt and not understand the caveats, etc etc. Some Microsoft employees have been calling for such limits for years [1, 2] [1] https://www.geekwire.com/2019/microsoft-github-workers-protest-ice-contracts-latest-demonstration-employee-activism/ https://www.geekwire.com/2019/microsoft-github-workers-prote... [2] https://www.cbsnews.com/news/microsoft-workers-asking-drop-police-contracts/ https://www.cbsnews.com/news/microsoft-workers-asking-drop-p...

chefandy 2y ago
If it's like what they did with the military, they're just trying to push them into a more expensive licensing scheme.
rockinghigh 2y ago
The NYPD has had a system built by Microsoft for 15 years: > Beginning in 2009, the NYPD, in partnership with Microsoft, built a powerful counterterrorism and policing tool called the Domain Awareness System (DAS). The DAS is a central platform used to aggregate data from internal and external closed-circuit television cameras (CCTV), license plate readers (LPRs), and environmental sensors, as well as 911 calls and other NYPD databases. The DAS uses an interactive dashboard interface to display real-time alerts whenever a 911 call is received or a sensor is triggered. The DAS also in- cludes mapping features that make it possible to survey and track targets. The DAS was originally built to support the Lower Manhattan Security Initiative (LMSI) – a public-private partnership – but has since expanded to cover the entire city, giving NYPD personnel direct access to thousands of cameras owned and operated by private orga- nizations. Until the development of the mobility platform and the mobile DAS system described below, the full capabilities of DAS have only been available to the Counterterrorism Bureau and a few other specially trained officers on desktop computers. As the NYPD upgrades its network, access to all DAS capabilities and resources will be expanded to all NYPD’s commands. https://www.nyc.gov/html/nypd/html/home/POA/pdf/Technology.pdf https://www.nyc.gov/html/nypd/html/home/POA/pdf/Technology.p...

deleted 2y ago

[deleted]

edot 2y ago

They are not banning them from using their AI tool. They're banning them from the currently-offered licensing structure. This is a profit-grab, not an ethical restriction. It is of the same vein as Sam Altman going to congress and spooking them in hopes of regulating AI so that they are the only ones allowed to build it. I wouldn't be shocked if we also saw a similar move to ban hospitals, schools, etc. from using the standard enterprise license under the guise of "we need to provide you with oversight for your vulnerable populations", and then charging them a hefty surcharge on top of the standard enterprise rate. "Responsible" AI is more profitable than standard AI - regardless of the nebulous nature of what "responsible" means.

jMyles 2y ago
Spot on. I'll explore whether this nit is worth picking: > "Responsible" AI is more profitable than standard AI - regardless of the nebulous nature of what "responsible" means. /s/regardless of/in part owing to

spxneo 2y ago

American law enforcement openly disregard the foundations from which the country was built upon yet pushes other countries to adopt ideas which it abandoned but won't admit it out of humiliation--that which we call "democracy" comes with the very hooks and strings it tells other countries to abandon. Just take a look at the countries that credulously adopted "democracy". They are not in a good shape and will probably never find itself out of the moving goal post that Western countries like to place on "lesser" countries. America is addicted to these 3 things: Oil, authoritarian labour, drugs. It's law enforcement is not meant to police those addictions at all but rather to ensure its continued uninterrupted supply even if it means violating its own principles that made the country great (about 40+ years ago). It's no wonder many Americans are yearning for past glory.

lupire 2y ago
Yes, the past glory of Legally mandated racial discrimination, and a war on Black people and Hispanics on a pretext of a War on Drugs.

deusum 2y ago

No one has pointed out that the AI companies are being sued for 'stealing' vast troves of copyrighted data, and the case is still in progress. If the police departments are complicit, then does copyright stand to lose de facto as AI will only be increasingly implemented while the case slogs through the courts?

kcb 2y ago
One bare minimum way to not get your data "stolen" is to not personally make it freely available on the internet.

flenserboy 2y ago

It is dangerous to allow groups of any sort to be excluded from licensing. This sort of behavior is far too easily weaponized against minority political, opinion, & affinity groups. While I'm not keen on the police having these tools in their hands, who else will be next? Who will be squeezed out of the economy because everyone who provides a particular sort of software or service decides to enact the same policies? This is why software should be sold, not licensed — a word processor should be treated no differently from a loaf of bread.

tehwebguy 2y ago
"First they came for the Police state and I said nothing..." lmao
deleted 2y ago
[deleted]

devindotcom 2y ago

FYI, Microsoft now says (we contacted before publishing) the policy we cited "contained an error" and was intended to only apply to facial recognition, and has been changed in the official code of conduct. We've updated the headline and body to reflect this change. A mod might want to change the HN hed too.

forgetfreeman 2y ago

[flagged]