13 ms·

> According to the court documents, the Fargo detective working the case then looked at Lipps' social media accounts and Tennessee driver's license photo. In hi

by whack 7mo ago

> According to the court documents, the Fargo detective working the case then looked at Lipps' social media accounts and Tennessee driver's license photo. In his charging document, the detective wrote that Lipps appeared to be the suspect based on facial features, body type and hairstyle and color. > Once they were in hand, Fargo police met with him and Lipps at the Cass County jail on Dec. 19. She had already been in jail for more than five months. It was the first time police interviewed her. How is this the fault of AI? It flagged a possible match. A live human detective confirmed it. And the criminal justice system, for reasons that have nothing to do with AI, let this woman sit in jail for 5 months before doing even interviewing her or doing any due diligence. There's a reason why we don't let AI autonomously jail people. Instead of scapegoating an AI bogeyman, maybe we should look instead at the professional human-in-the-loop who shirked all responsibility, and a criminal justice system that thinks it is okay to jail people for 5 months before even starting to assess their guilt.

RobRivera 7mo ago
I think it's more nuanced; it is one error in a Tragedy of Errors.
- throwaway17_17 7mo ago
  This was not a series of errors, this is (as a statistical inference) the system working as designed. This is not uncommon, it is not unplanned. The extradition of suspects from State to State is designed legislatively to function this way. I also think there is more nuance to this situation than AI bad // Human Bad :: choose one. But while a tragedy, the ‘correct’ functioning of a system that produces tragedy doesn’t make that function and error.
themafia 7mo ago
> How is this the fault of AI? It could be the fault of the company that's selling this service. They often make wildly inaccurate claims about the utility and accuracy of their systems. [0] > There's a reason why we don't let AI autonomously jail people. Yes we do. [1] > and a criminal justice system that thinks it is okay to jail people for 5 months before even starting to assess their guilt. Her guilt was assessed. That's why she had no bail. It assessed it incorrectly, but the error is more complicated than your reaction implies. [0]: https://thisisreno.com/2026/03/lawsuit-reno-police-ai-policing/ https://thisisreno.com/2026/03/lawsuit-reno-police-ai-polici... [1]: https://projects.tampabay.com/projects/2020/investigations/police-pasco-sheriff-targeted/intelligence-led-policing/ https://projects.tampabay.com/projects/2020/investigations/p...
- throwaway17_17 7mo ago
  To clarify one point, her not having a bail is a function of the way interstate ‘fugitive’ warrants are designed. The Court in Tennessee had no ability to set bail, and until she entered the physical custody of North Dakota she can not have bail set. Also, her guilt was not assessed in any common meaning of the term. The requirement for holding a person in custody, with or without bail, is probable cause. The only thing assessed was did law enforcement present a statement to a Judge that was possible to be believed in the light most favorable to the prosecution.
rglover 7mo ago
> How is this the fault of AI? It flagged a possible match. A live human detective confirmed it. Because we're seeing the first instances of what reality looks like with AI in the hands of the average bear. Just like the excuse was "but the computer said it was correct," now we're just shifting to "but the AI said it was correct." Don't underestimate how much authority and thinking people will delegate to machines. Not to mention the lengths they'll go to weasel out of taking responsibility for a screw up like this (saw another comment in this thread about the Chief of Police stepping down but it being framed as "retirement").
- pj_mukh 7mo ago
  I'm sorry but this is a piss-poor excuse. When I Claude code broken features, I'm responsible 100%. Why are cops not treated the same way? OP is right, AI is totally irrelevant in this story. If the point is "cops can't be trusted". Why do they have GUNS?! AI is the least of your problems. I feel like I'm going crazy with this narrative.
  - wat10000 7mo ago
    When are cops ever treated the same way as the rest of us?
    - deepsun 7mo ago
      Well in most cases I would prefer to have a cop's word to outweigh a word of an average joe.
      - wat10000 7mo ago
        Why should having that particular job give you that privilege? All should be equal before the law.
      - amanaplanacanal 7mo ago
        Do you think police are inherently more honest than everybody else? Why would you think that?
obviouslynotme 7mo ago
It's not. This is just an acceleration in the unraveling of society facilitated by AI. As someone whose childhood included so many "robots will kill humans" books and movies, I am flabbergasted that the AI apocalypse will be dumb humans overtrusting faulty AI in important matters until everything falls apart. Most humans cannot distinguish AI from actual intelligence. When you combine that with bureaucrats innate tendency to say, "Computer said so," you end up with bizarre situations like this. If a person had made this facial match, another human would have relentlessly jeered him. Since a computer running AI did it, no one even cared to think about it. Computers are wildly dangerous, not because of anything innate but because of how humans act around them.
- throwaway173738 7mo ago
  > It's not. This is just an acceleration in the unraveling of society facilitated by AI. As someone whose childhood included so many "robots will kill humans" books and movies, I am flabbergasted that the AI apocalypse will be dumb humans overtrusting faulty AI in important matters until everything falls apart. This is literally the plot of most of those books and the way they differ is in how everything falls apart. In some of them the AI supplants us entirely and kills us all. In others it gets taught to kill us all. In others it gets really good at giving us what we ask for until everything falls apart. But it’s taken as a given that unless we change something innate in our culture AI will be our downfall.
- fc417fc802 7mo ago
  > If a person had made this facial match, another human would have relentlessly jeered him. The glaringly obvious problem here is that our justice system should not be constructed in such a way so as to be reliant on someone's coworker shaming him. That is not a sensible check against a systemic failure. We're supposed to have due process. If someone skips or otherwise subverts due process the justifications don't matter. The root issue is that due process was skipped. Why was that even possible to begin with?
blitzar 7mo ago
computer said yes
- discordance 7mo ago
  lgtm
caconym_ 7mo ago
This particular "AI bogeyman" isn't just AI; it's cops with AI and in particular cops with facial recognition tools, dragnet LPR surveillance tools, and all this other new technology that essentially picks somebody's name out of a hat to have their life temporarily (or [semi-]permanently) ruined by shithead cops who won't ever face any real accountability. This keeps happening, and the reason it keeps happening is that shithead cops have these tools and are using them. Until we can find a reliable way to prevent this from happening, which may or may not be possible, cops who may or may not be shitheads should not have access to these tools.
- throwaway314155 7mo ago
  There’s nothing wrong with your comment per se, but it’s almost as if you didn’t even read the comment you’re responding to.
  - Retric 7mo ago
    Seems like a direct response to me. >> How is this the fault of AI? > This particular "AI bogeyman" isn't just AI; it's cops with AI You can’t separate the thing from how it will be used. It’s like arguing that cars on their own aren’t particularly dangerous, but the point of buying a car is to use it thus risking the general public.
    - fc417fc802 7mo ago
      But you can in fact argue exactly that. If (arbitrary example) pedestrians are being killed due to poor road engineering practices it isn't reasonable to point at cars and say "see those are the root problem" when in fact it's due to a willful lack of sidewalks or marked crossings or whatever. Being adjacent to something bad doesn't equate to being the root cause.
      - Retric 7mo ago
        History shows the timeline of dependence here. Before the introduction of cars, “poor road engineering practices” wouldn’t result in those deaths. So clearly it’s cars that are necessitating sidewalks, etc. Same deal here, if something “becomes a problem” because of the introduction of AI, it’s AI that is the root case of the resulting issues. Many people are tempted to argue that flawed humans can’t implement the perfect system that is Anarchy, Communism, Recycling programs, or whatever but treating systems as needing to operate on the real world is productive where complaining about humans isn’t.
deleted 7mo ago
[deleted]
rightbyte 7mo ago
> How is this the fault of AI? The false positive rate combined with scanning millions of pictures might make the chance of arresting the wrong person really high.
- akimbostrawman 7mo ago
  The wrong detection is the AI's fault. Anything after and based off that is the fault of humans.
  - rectang 7mo ago
    Before the misuse, there is opportunity to predict that misuse is certain to occur.
stego-tech 7mo ago
It's the fault of the tool because our society treats the tools as superior judgements than humans and to be trusted completely as a means of deflecting accountability - something any and every minority group has been warning about for fucking decades. The reason everyone rushes to defend the tool's use is because holding humans accountable would mean throwing these tools out entirely in most cases, due to internal human biases and a decline in basic critical and cognitive thinking skills. The marketing has been the same since the 80s: the tool is superior (until it isn't), the tool shall be trusted completely (until it fails), the tool cannot make mistakes (until it does). If folks actually listened to the victims of this shit, companies like Flock and Palantir would be gutted and their founders barred from any sort of office of responsibility, at minimum. The fact so many deflect blame from the tool like the marketing manual demands shows they don't actually give a shit about the humans wrapped up in the harms, or the misuse and misappropriation of these tools by persons wholly unaccountable under the law, but only about defending a shiny thing they personally like.
- pixl97 7mo ago
  >rushes to defend the tool's use is because holding humans accountable would mean throwing these tools out entirely in most cases, due to internal human biases and a decline in basic critical and cognitive thinking skill The magical past where people had critical thinking skills never existed. We put a lot of trust in tools is because people are unfucking reliable. Hence why in most cases actual physical evidence does a far better job than witness testimony. This said, people are lazy. It is one of our greatest and worst traits. When we are allowed to be lazy, especially with tools bad things happen.
type0 7mo ago
> Instead of scapegoating an AI bogeyman One big reason for AI adoption everywhere is that you can use it as a scapegoat
antonymoose 7mo ago
Reminds me of a case that just popped up in my neck of the woods. Man gets pulled over on an expired plate. They search based on this fact, find a pill bottle (for Irritable Bowel Syndrome) and magically find he’s trafficking cocaine and fentanyl. Months later a lab test exonerates the poor guy. https://www.wyff4.com/article/deputies-falsely-identify-ibs-medication-as-drugs-lawsuit/70715236 https://www.wyff4.com/article/deputies-falsely-identify-ibs-...
- bloomingeek 7mo ago
  I've always maintained one of the worst things that can happen to you is sitting in court before a jury of your peers, because most can't comprehend the meaning of the law outside of their feelings. NOW the worst thing is having yourself in the hands of cops who just don't give a damn or became a cop for the use of power.
- JuniperMesos 7mo ago
  This one seems pretty reasonable - according to the article, the cops pulled him over for swerving lanes (driving unsafely on public roads in a reasonable thing to want to police), and then discovered that he was driving on a suspended license, which he admitted to (it's reasonable to have a system for suspending peoples' drivers licenses that is enforced by the police). The police find the pill bottle and don't believe him when he tells them it's a legitimate drug, then "conduct[..] multiple field drug tests, which produced a positive result for fentanyl. Getchius was taken into custody and transported to the Greenwood County Detention Center. Shortly after, another drug test was completed and returned positive results for cocaine." So it wasn't just the pill bottle, it was multiple other drug tests. I think you could make a reasonable argument that drug use shouldn't constitute a crime in and of itself - although it probably should if you're driving a car, for legitimate traffic safety reasons, I don't find DUI laws objectionable. Or you could make an argument that the criminal justice system shouldn't interfere with peoples' decision to use and sell drugs. I'm sympathetic to this myself, but I think especially in the case of opioids like fentanyl, the situation where government paternalism makes it illegal to sell opioids probably discourages enough destructive use of these drugs by unwise or already-addicted people that it's still net-positive in terms of human welfare. I suspect a society where it was simply legal to use and sell opioids would have a lot more human suffering in it than our own (possibly because in the absence of laws banning open opioid dealing, people who are close to severe opioid addicts might simply commit vigilante murders of suspected opioid dealers, and be left unconvicted by sympathetic juries). And once you hold the position that it's legitimate for the government to legally restrict the sale and use of these drugs, then you necessarily have to have something like police and something like a criminal justice system that investigates whether a person might be actually using and selling opioids and then lying about it. The fact that the guy was in fact once addicted to some drug and "was working at rehab and addiction centers in Florida at the time of his arrest." is additional evidence that he might have returned to drug use, and there's no way to make cops who investigate opioid-related crimes not think this.
ares623 7mo ago
I hope you take this as a teaching/learning opportunity
worik 7mo ago
> How is this the fault of AI? It is not. It is the fault of the police AI models are tools. When mistakes are made they are the mistake of the operator of said tool This AI model was badly misused, this woman should get a metric shit tonne of compensation, but it was the fault of the police.
culi 7mo ago
Study after study has shown a very strong and consistent bias of humans to trust "automated systems" in face of any ambiguity
lokar 7mo ago
Automation has a strong tendency to degrade diligence. I see this all the time in operational / production settings. Having a loop with automation reviewed and approved by a human degrades very fast. I only approve automation that has a quick path to unsupervised operation.
unethical_ban 7mo ago
Someone from the government should be in jail for this kind of oversight.
- throwaway173738 7mo ago
  I think the taxpayers owe this lady at least a couple million if not more for the inconvenience they chose to put her through.
  - throwaway17_17 7mo ago
    I agree, but our system doesn’t value things that way. Texas, which is one of the highest paying States for cases where intentional, fraudulent, or grossly negligent actions result in wrongful incarceration pay $80,000 dollars per year a person is locked up. But the caveat is that time only starts counting after you are sentenced, so wouldn’t even apply in TFA’s case.
    - dghlsakjg 7mo ago
      That's statutory damages on a wrongful conviction. This is an entirely different situation, and the amount would not be statutory.
  - subscribed 7mo ago
    That shouldn't be taxpayers. That should come from the malpractice insurance, similarly how doctors pay. After a while, when premiums shoot up, they'd start to behave
    - sieabahlpark 7mo ago
      [dead]
    - selbyk 7mo ago
      Who pays the insurance premiums?
- 7mo ago
deleted 7mo ago
[deleted]
danso 7mo ago
> How is this the fault of AI? AI is being used by bureaucrats and enforcers to justify lazy, harmful conclusions. You don't live in the real world if you think "just punish the bureaucrats, don't make it about AI" is going to remotely rectify this toxic feedback loop and ecosystem.
- fc417fc802 7mo ago
  No, we definitely should punish bureaucrats and enforcers who act negligently. If someone in a position of authority flagrantly fails to do his job and it directly harms someone he should be held accountable. That would provide a strong incentive for future actors to take their responsibilities seriously. If an engineer signs off on an obviously faulty building plan and people die as a result we hold him accountable. This is no different.
dragonwriter 7mo ago
> How is this the fault of AI It isn't, the article doesn't claim (or even imply) that it is "the fault" of AI, only that AI was part of the chain of events, and nothing is the fault of AI until AI is sufficiently advanced to constitute a moral actor. “At the source of every error which is blamed on the computer, you will find at least two human errors, one of which is the error of blaming it on the computer” remains true. OTOH, it is potentially the fault of the reliance human actors put on an AI determination.
petre 7mo ago
> How is this the fault of AI? Humans being human. Getting lazy, being incompetent, getting incompetent with AI use or simply being biased. The wrongfully arrested person doesn't even resamble the perpetrator. Maybe if they were held accountable forthese actions, they would act responsibly?
croes 7mo ago
Where does it say that AI is blamed. It says she was misidentified using facial recognition. That’s exactly what happened
plorg 7mo ago
I think the biggest problem is that the popular narratives about AI enable this like of accountability sink.
- hdgvhicv 7mo ago
  Before AI it was outsourcing. “Not my fault the system is down and we’re losing 1m an hour, AWS is having a bad day”
EchoReflection 7mo ago
100% 100% 100% humanity is so obsessed with ai that we're losing...our humanity. "blame the mindless, soulless robots! how could we have possibly known that they need to be supervised?! aren't they basically just humans that don't need to rest or eat?"
arwhatever 7mo ago
Devils advocate: what if a facial recognition system with a large enough database can always find an unrelated/innocent person that looks similar enough to convince the human?
vbezhenar 7mo ago
At this point I think that AI will perform human duties better than human. So probably it's better to let AI autonomously jail people, of course with all the necessary procedures as required by law.
- rectang 7mo ago
  As a work of literature, I applaud your Swiftian “modest proposal”.
Nevermark 7mo ago
If many people's writing skills are suffering, due to highly convenient AI support, just imagine how fast mediocre crime investigation skills are going to devolve. It is going to get bad in every skilled area of human managed bureaucracy. The number of legal filings found to include AI confabulations is just the obvious surface.
jjav 7mo ago
> A live human detective confirmed it. I doubt it, due to human nature. Perhaps the process says the human must consciously validate, but a lot of humans in many cases will just rubberstamp what the AI said. That's the risk.
rolandog 7mo ago
It is the fault of the coders, the salespeople who over-promised the capabilities of the system, the lawmakers who have not regulated or demanded a minimum percentage of accuracy from those products, the AI' company's onboarding trainers, the cops that were trained to use the software, the jailers, and maybe other related positions that should've taken a better interest in making a better system, not a more cruel one
dyauspitr 7mo ago
Because if you let this slide the human, such as he is, will be removed from the loop and these mistakes will become acceptable once departments get used to how cheap the AI is compared to a human. There will be no going back and mistakes like this will just become accepted collateral damage.
deleted 7mo ago
[deleted]
tgv 7mo ago
> How is this the fault of AI? I'll reply to the top of the discussion too: it's because it was purely made for this purpose. There's no use for it outside surveillance. And it's not even good enough. It's only purpose is checking boxes and transferring money. Miscarriage of justice is an unfortunate, but calculated side effect.
tim-tday 7mo ago
The legal system has long treated a computer match as infallible. This has led to miscarriage of justice on a grand scale.

by whack 7mo ago

RobRivera 7mo ago

I think it's more nuanced; it is one error in a Tragedy of Errors.

throwaway17_17 7mo ago
This was not a series of errors, this is (as a statistical inference) the system working as designed. This is not uncommon, it is not unplanned. The extradition of suspects from State to State is designed legislatively to function this way. I also think there is more nuance to this situation than AI bad // Human Bad :: choose one. But while a tragedy, the ‘correct’ functioning of a system that produces tragedy doesn’t make that function and error.

themafia 7mo ago

> How is this the fault of AI? It could be the fault of the company that's selling this service. They often make wildly inaccurate claims about the utility and accuracy of their systems. [0] > There's a reason why we don't let AI autonomously jail people. Yes we do. [1] > and a criminal justice system that thinks it is okay to jail people for 5 months before even starting to assess their guilt. Her guilt was assessed. That's why she had no bail. It assessed it incorrectly, but the error is more complicated than your reaction implies. [0]: https://thisisreno.com/2026/03/lawsuit-reno-police-ai-policing/ https://thisisreno.com/2026/03/lawsuit-reno-police-ai-polici... [1]: https://projects.tampabay.com/projects/2020/investigations/police-pasco-sheriff-targeted/intelligence-led-policing/ https://projects.tampabay.com/projects/2020/investigations/p...

throwaway17_17 7mo ago
To clarify one point, her not having a bail is a function of the way interstate ‘fugitive’ warrants are designed. The Court in Tennessee had no ability to set bail, and until she entered the physical custody of North Dakota she can not have bail set. Also, her guilt was not assessed in any common meaning of the term. The requirement for holding a person in custody, with or without bail, is probable cause. The only thing assessed was did law enforcement present a statement to a Judge that was possible to be believed in the light most favorable to the prosecution.

rglover 7mo ago

> How is this the fault of AI? It flagged a possible match. A live human detective confirmed it. Because we're seeing the first instances of what reality looks like with AI in the hands of the average bear. Just like the excuse was "but the computer said it was correct," now we're just shifting to "but the AI said it was correct." Don't underestimate how much authority and thinking people will delegate to machines. Not to mention the lengths they'll go to weasel out of taking responsibility for a screw up like this (saw another comment in this thread about the Chief of Police stepping down but it being framed as "retirement").

pj_mukh 7mo ago
I'm sorry but this is a piss-poor excuse. When I Claude code broken features, I'm responsible 100%. Why are cops not treated the same way? OP is right, AI is totally irrelevant in this story. If the point is "cops can't be trusted". Why do they have GUNS?! AI is the least of your problems. I feel like I'm going crazy with this narrative.
- wat10000 7mo ago
  When are cops ever treated the same way as the rest of us?
  - deepsun 7mo ago
    Well in most cases I would prefer to have a cop's word to outweigh a word of an average joe.
    - wat10000 7mo ago
      Why should having that particular job give you that privilege? All should be equal before the law.
    - amanaplanacanal 7mo ago
      Do you think police are inherently more honest than everybody else? Why would you think that?

obviouslynotme 7mo ago

It's not. This is just an acceleration in the unraveling of society facilitated by AI. As someone whose childhood included so many "robots will kill humans" books and movies, I am flabbergasted that the AI apocalypse will be dumb humans overtrusting faulty AI in important matters until everything falls apart. Most humans cannot distinguish AI from actual intelligence. When you combine that with bureaucrats innate tendency to say, "Computer said so," you end up with bizarre situations like this. If a person had made this facial match, another human would have relentlessly jeered him. Since a computer running AI did it, no one even cared to think about it. Computers are wildly dangerous, not because of anything innate but because of how humans act around them.

throwaway173738 7mo ago
> It's not. This is just an acceleration in the unraveling of society facilitated by AI. As someone whose childhood included so many "robots will kill humans" books and movies, I am flabbergasted that the AI apocalypse will be dumb humans overtrusting faulty AI in important matters until everything falls apart. This is literally the plot of most of those books and the way they differ is in how everything falls apart. In some of them the AI supplants us entirely and kills us all. In others it gets taught to kill us all. In others it gets really good at giving us what we ask for until everything falls apart. But it’s taken as a given that unless we change something innate in our culture AI will be our downfall.
fc417fc802 7mo ago
> If a person had made this facial match, another human would have relentlessly jeered him. The glaringly obvious problem here is that our justice system should not be constructed in such a way so as to be reliant on someone's coworker shaming him. That is not a sensible check against a systemic failure. We're supposed to have due process. If someone skips or otherwise subverts due process the justifications don't matter. The root issue is that due process was skipped. Why was that even possible to begin with?

blitzar 7mo ago

computer said yes

discordance 7mo ago
lgtm

caconym_ 7mo ago

This particular "AI bogeyman" isn't just AI; it's cops with AI and in particular cops with facial recognition tools, dragnet LPR surveillance tools, and all this other new technology that essentially picks somebody's name out of a hat to have their life temporarily (or [semi-]permanently) ruined by shithead cops who won't ever face any real accountability. This keeps happening, and the reason it keeps happening is that shithead cops have these tools and are using them. Until we can find a reliable way to prevent this from happening, which may or may not be possible, cops who may or may not be shitheads should not have access to these tools.

throwaway314155 7mo ago
There’s nothing wrong with your comment per se, but it’s almost as if you didn’t even read the comment you’re responding to.
- Retric 7mo ago
  Seems like a direct response to me. >> How is this the fault of AI? > This particular "AI bogeyman" isn't just AI; it's cops with AI You can’t separate the thing from how it will be used. It’s like arguing that cars on their own aren’t particularly dangerous, but the point of buying a car is to use it thus risking the general public.
  - fc417fc802 7mo ago
    But you can in fact argue exactly that. If (arbitrary example) pedestrians are being killed due to poor road engineering practices it isn't reasonable to point at cars and say "see those are the root problem" when in fact it's due to a willful lack of sidewalks or marked crossings or whatever. Being adjacent to something bad doesn't equate to being the root cause.
    - Retric 7mo ago
      History shows the timeline of dependence here. Before the introduction of cars, “poor road engineering practices” wouldn’t result in those deaths. So clearly it’s cars that are necessitating sidewalks, etc. Same deal here, if something “becomes a problem” because of the introduction of AI, it’s AI that is the root case of the resulting issues. Many people are tempted to argue that flawed humans can’t implement the perfect system that is Anarchy, Communism, Recycling programs, or whatever but treating systems as needing to operate on the real world is productive where complaining about humans isn’t.

deleted 7mo ago

[deleted]

rightbyte 7mo ago

> How is this the fault of AI? The false positive rate combined with scanning millions of pictures might make the chance of arresting the wrong person really high.

akimbostrawman 7mo ago
The wrong detection is the AI's fault. Anything after and based off that is the fault of humans.
- rectang 7mo ago
  Before the misuse, there is opportunity to predict that misuse is certain to occur.

stego-tech 7mo ago

It's the fault of the tool because our society treats the tools as superior judgements than humans and to be trusted completely as a means of deflecting accountability - something any and every minority group has been warning about for fucking decades. The reason everyone rushes to defend the tool's use is because holding humans accountable would mean throwing these tools out entirely in most cases, due to internal human biases and a decline in basic critical and cognitive thinking skills. The marketing has been the same since the 80s: the tool is superior (until it isn't), the tool shall be trusted completely (until it fails), the tool cannot make mistakes (until it does). If folks actually listened to the victims of this shit, companies like Flock and Palantir would be gutted and their founders barred from any sort of office of responsibility, at minimum. The fact so many deflect blame from the tool like the marketing manual demands shows they don't actually give a shit about the humans wrapped up in the harms, or the misuse and misappropriation of these tools by persons wholly unaccountable under the law, but only about defending a shiny thing they personally like.

pixl97 7mo ago
>rushes to defend the tool's use is because holding humans accountable would mean throwing these tools out entirely in most cases, due to internal human biases and a decline in basic critical and cognitive thinking skill The magical past where people had critical thinking skills never existed. We put a lot of trust in tools is because people are unfucking reliable. Hence why in most cases actual physical evidence does a far better job than witness testimony. This said, people are lazy. It is one of our greatest and worst traits. When we are allowed to be lazy, especially with tools bad things happen.

type0 7mo ago

> Instead of scapegoating an AI bogeyman One big reason for AI adoption everywhere is that you can use it as a scapegoat

antonymoose 7mo ago

Reminds me of a case that just popped up in my neck of the woods. Man gets pulled over on an expired plate. They search based on this fact, find a pill bottle (for Irritable Bowel Syndrome) and magically find he’s trafficking cocaine and fentanyl. Months later a lab test exonerates the poor guy. https://www.wyff4.com/article/deputies-falsely-identify-ibs-medication-as-drugs-lawsuit/70715236 https://www.wyff4.com/article/deputies-falsely-identify-ibs-...

bloomingeek 7mo ago
I've always maintained one of the worst things that can happen to you is sitting in court before a jury of your peers, because most can't comprehend the meaning of the law outside of their feelings. NOW the worst thing is having yourself in the hands of cops who just don't give a damn or became a cop for the use of power.
JuniperMesos 7mo ago
This one seems pretty reasonable - according to the article, the cops pulled him over for swerving lanes (driving unsafely on public roads in a reasonable thing to want to police), and then discovered that he was driving on a suspended license, which he admitted to (it's reasonable to have a system for suspending peoples' drivers licenses that is enforced by the police). The police find the pill bottle and don't believe him when he tells them it's a legitimate drug, then "conduct[..] multiple field drug tests, which produced a positive result for fentanyl. Getchius was taken into custody and transported to the Greenwood County Detention Center. Shortly after, another drug test was completed and returned positive results for cocaine." So it wasn't just the pill bottle, it was multiple other drug tests. I think you could make a reasonable argument that drug use shouldn't constitute a crime in and of itself - although it probably should if you're driving a car, for legitimate traffic safety reasons, I don't find DUI laws objectionable. Or you could make an argument that the criminal justice system shouldn't interfere with peoples' decision to use and sell drugs. I'm sympathetic to this myself, but I think especially in the case of opioids like fentanyl, the situation where government paternalism makes it illegal to sell opioids probably discourages enough destructive use of these drugs by unwise or already-addicted people that it's still net-positive in terms of human welfare. I suspect a society where it was simply legal to use and sell opioids would have a lot more human suffering in it than our own (possibly because in the absence of laws banning open opioid dealing, people who are close to severe opioid addicts might simply commit vigilante murders of suspected opioid dealers, and be left unconvicted by sympathetic juries). And once you hold the position that it's legitimate for the government to legally restrict the sale and use of these drugs, then you necessarily have to have something like police and something like a criminal justice system that investigates whether a person might be actually using and selling opioids and then lying about it. The fact that the guy was in fact once addicted to some drug and "was working at rehab and addiction centers in Florida at the time of his arrest." is additional evidence that he might have returned to drug use, and there's no way to make cops who investigate opioid-related crimes not think this.

ares623 7mo ago

I hope you take this as a teaching/learning opportunity

worik 7mo ago

> How is this the fault of AI? It is not. It is the fault of the police AI models are tools. When mistakes are made they are the mistake of the operator of said tool This AI model was badly misused, this woman should get a metric shit tonne of compensation, but it was the fault of the police.

culi 7mo ago

Study after study has shown a very strong and consistent bias of humans to trust "automated systems" in face of any ambiguity

lokar 7mo ago

Automation has a strong tendency to degrade diligence. I see this all the time in operational / production settings. Having a loop with automation reviewed and approved by a human degrades very fast. I only approve automation that has a quick path to unsupervised operation.

unethical_ban 7mo ago

Someone from the government should be in jail for this kind of oversight.

throwaway173738 7mo ago
I think the taxpayers owe this lady at least a couple million if not more for the inconvenience they chose to put her through.
- throwaway17_17 7mo ago
  I agree, but our system doesn’t value things that way. Texas, which is one of the highest paying States for cases where intentional, fraudulent, or grossly negligent actions result in wrongful incarceration pay $80,000 dollars per year a person is locked up. But the caveat is that time only starts counting after you are sentenced, so wouldn’t even apply in TFA’s case.
  - dghlsakjg 7mo ago
    That's statutory damages on a wrongful conviction. This is an entirely different situation, and the amount would not be statutory.
- subscribed 7mo ago
  That shouldn't be taxpayers. That should come from the malpractice insurance, similarly how doctors pay. After a while, when premiums shoot up, they'd start to behave
  - sieabahlpark 7mo ago
    [dead]
  - selbyk 7mo ago
    Who pays the insurance premiums?
7mo ago

deleted 7mo ago

[deleted]

danso 7mo ago

> How is this the fault of AI? AI is being used by bureaucrats and enforcers to justify lazy, harmful conclusions. You don't live in the real world if you think "just punish the bureaucrats, don't make it about AI" is going to remotely rectify this toxic feedback loop and ecosystem.

fc417fc802 7mo ago
No, we definitely should punish bureaucrats and enforcers who act negligently. If someone in a position of authority flagrantly fails to do his job and it directly harms someone he should be held accountable. That would provide a strong incentive for future actors to take their responsibilities seriously. If an engineer signs off on an obviously faulty building plan and people die as a result we hold him accountable. This is no different.

dragonwriter 7mo ago

> How is this the fault of AI It isn't, the article doesn't claim (or even imply) that it is "the fault" of AI, only that AI was part of the chain of events, and nothing is the fault of AI until AI is sufficiently advanced to constitute a moral actor. “At the source of every error which is blamed on the computer, you will find at least two human errors, one of which is the error of blaming it on the computer” remains true. OTOH, it is potentially the fault of the reliance human actors put on an AI determination.

petre 7mo ago

> How is this the fault of AI? Humans being human. Getting lazy, being incompetent, getting incompetent with AI use or simply being biased. The wrongfully arrested person doesn't even resamble the perpetrator. Maybe if they were held accountable forthese actions, they would act responsibly?

croes 7mo ago

Where does it say that AI is blamed. It says she was misidentified using facial recognition. That’s exactly what happened

plorg 7mo ago

I think the biggest problem is that the popular narratives about AI enable this like of accountability sink.

hdgvhicv 7mo ago
Before AI it was outsourcing. “Not my fault the system is down and we’re losing 1m an hour, AWS is having a bad day”

EchoReflection 7mo ago

100% 100% 100% humanity is so obsessed with ai that we're losing...our humanity. "blame the mindless, soulless robots! how could we have possibly known that they need to be supervised?! aren't they basically just humans that don't need to rest or eat?"

arwhatever 7mo ago

Devils advocate: what if a facial recognition system with a large enough database can always find an unrelated/innocent person that looks similar enough to convince the human?

vbezhenar 7mo ago

At this point I think that AI will perform human duties better than human. So probably it's better to let AI autonomously jail people, of course with all the necessary procedures as required by law.

rectang 7mo ago
As a work of literature, I applaud your Swiftian “modest proposal”.

Nevermark 7mo ago

If many people's writing skills are suffering, due to highly convenient AI support, just imagine how fast mediocre crime investigation skills are going to devolve. It is going to get bad in every skilled area of human managed bureaucracy. The number of legal filings found to include AI confabulations is just the obvious surface.

jjav 7mo ago

> A live human detective confirmed it. I doubt it, due to human nature. Perhaps the process says the human must consciously validate, but a lot of humans in many cases will just rubberstamp what the AI said. That's the risk.

rolandog 7mo ago

It is the fault of the coders, the salespeople who over-promised the capabilities of the system, the lawmakers who have not regulated or demanded a minimum percentage of accuracy from those products, the AI' company's onboarding trainers, the cops that were trained to use the software, the jailers, and maybe other related positions that should've taken a better interest in making a better system, not a more cruel one

dyauspitr 7mo ago

Because if you let this slide the human, such as he is, will be removed from the loop and these mistakes will become acceptable once departments get used to how cheap the AI is compared to a human. There will be no going back and mistakes like this will just become accepted collateral damage.

deleted 7mo ago

[deleted]

tgv 7mo ago

> How is this the fault of AI? I'll reply to the top of the discussion too: it's because it was purely made for this purpose. There's no use for it outside surveillance. And it's not even good enough. It's only purpose is checking boxes and transferring money. Miscarriage of justice is an unfortunate, but calculated side effect.

tim-tday 7mo ago

The legal system has long treated a computer match as infallible. This has led to miscarriage of justice on a grand scale.