13 ms·

A man spent a year in jail on murder charge that hinged on disputed AI evidence

372 points by LinuxBender 5y ago

the-dude 5y ago
This is about ShotSpotter. https://hn.algolia.com/?q=ShotSpotter https://hn.algolia.com/?q=ShotSpotter
PragmaticPulp 5y ago
The article says that employees of the AI company (ShotSpotter) manually reviewed and classified the sounds as gunshots: > records showed that ShotSpotter actually initially picked up what sounded like a firework a mile away, and this was later reclassified by ShotSpotter staff to be a gunshot at the intersection where and when Williams was seen on camera. So the AI didn’t even make the call. The staff did, manually. I assume that means the actual audio is available and entered into evidence? If humans are making the call then blaming AI seems like a stretch. That’s almost like blaming the motion detection algorithms for triggering video recordings that were later reviewed by humans. It’s still humans reviewing the recordings and making decisions.
- jtbayly 5y ago
  It may not be right to blame AI, but it's certainly the case that they were presenting it as "the system says there was a shot fired here." The bigger point is that the "system" can't be trusted. And the fact that they need to manually classify things is proof of that fact. (It's way more complicated than just an audio recording. It's recordings from numerous locations, attempting to triangulate the original location, without being confused by echoes and other things.)
  - PragmaticPulp 5y ago
    > It may not be right to blame AI, but it's certainly the case that they were presenting it as "the system says there was a shot fired here." I think a crucial distinction is to clarify who “they” is: Who was making the claim? From the article, it seems that the prosecutors may have been the ones exaggerating the evidence: > Prosecutors said ShotSpotter picked up a gunshot sound where Williams was seen on surveillance camera footage in his car, putting it all forward as proof that Williams shot Herring right there and then. Naturally the defense wanted to see the actual evidence, so that’s what they got down to. I know everyone wants to vilify AI, but I think the actual bad actor in this article might be the prosecutor trying to misrepresent the evidence.
    - Aerroon 5y ago
      >I think a crucial distinction is to clarify who “they” is: Who was making the claim? From the article, it seems that the prosecutors may have been the ones exaggerating the evidence: The prosecutors are the "they". The further removed you get from people who know the 'AI' the more the validity of the evidence provided by the AI relies merely on what others have said. The people working on the system know that it can be faulty. Other software developers have an idea that it could be faulty. Prosecutors are told that it's alright. They turn around and argue it in a court of law. And the further you get the less people have any idea that the system is faulty. Down the line people will go "the machine says so, therefore it must be so".
      - ClumsyPilot 5y ago
        >Down the line people will go "the machine says so, therefore it must be so". Thats the real feature of AI, devision laundering: a flawed decision made by someone for their own gain now appears impartial and trustworthy, and come wothout accountability
    - jtbayly 5y ago
      Sure. But the point is that they misrepresent the evidence by pointing to the system as trustworthy. Even without tampering with the evidence the system isn’t trustworthy beyond a reasonable shadow of a doubt.
- stefan_ 5y ago
  That's the point? These systems invariably turn out to not be any sort of AI at all, but sales people ready to make a call any way their customer (police) wants it made. Later it's presented as AI to a jury. It's basically the same scam as lots of forensics, junk science to get a desired result.
  - giantg2 5y ago
    I generally agree, but wouldn't necessarily say a lot of forensics is junk science. The science is generally sound (procedures and theory). Usually it's the prosecutors that present the forensics in a way that misrepresents what the results actually mean. Or lab techs make mistakes or perform misconduct (How to Fix a Drug Scandal on Netflix is interesting).
    - wyager 5y ago
      Fingerprinting is junk science. Polygraphs are junk science. Tons of things that have been or are admissible as evidence are junk science.
      - giantg2 5y ago
        I wouldn't call polygraphs forensics. Most states do not allow them as evidence. Do you have a link for the fingerprinting claim?
        monetus 5y ago
        (Not who you responded to) https://www.crammlawfirm.com/fingerprint-match-jails-innocent-man/ https://www.crammlawfirm.com/fingerprint-match-jails-innocen... This isn't the case I originally recalled, and doesn't say how many points were matched, but I remember a 5 point match being used to convict a person who was later proven innocent.
        newswasboring 5y ago
        That shows its flawed, not junk.
- helsinkiandrew 5y ago
  It's scary that prosecutors thought that the ShotSpotter evidence is suitable for a court. It's amazing technology that can help police get to a crime scene quickly but is far too easy to spoof or get the wrong answer due to reflections and other sounds etc. Hopefully cases like this will eventually stop the system being used in court.
  - SamoyedFurFluff 5y ago
    I mean, drug dogs are still a thing even though we know they tend to respond to owners desires regardless of if there’s drugs present.
    - yawaworht1978 5y ago
      Wow i have never heard of that. Curious to see a link/article on the issue, if you can provide.
      - DaftDank 5y ago
        This is a decent summary of court cases related to the dogs: https://www.nlrg.com/criminal-law-legal-research/bid/84670/CRIMINAL-LAW-UPDATE-Reliability-of-Narcotics-Dogs-to-Be-Revisited-by-U-S-Supreme-Court https://www.nlrg.com/criminal-law-legal-research/bid/84670/C... I think this part is pretty key, and has always been a big open question in my mind: "The court further recognized a potential problem in assessing reliability related to handler cuing, as even well-trained dogs could respond to subconscious cues from their handlers. For this reason, the court determined that a critical factor in determining reliability is the record of false positive alerts made by the dog." The subconscious cues, or even deliberate cues that only the handler and dog know, could allow an officer with ill intentions to make some kind of signal only known to him and the dog, which causes the dog to make an "alert", giving the cop probable cause.
        leephillips 5y ago
        Some departments deploy dogs that alert on all vehicles when they want an excuse to search. In at least one department, this was referred to as “probable cause on four legs”. If judges continue to allow this quality of evidence, the police will continue to abuse the legal environment that creates.
- nerdponx 5y ago
  > Crucially, Williams' lawyers – public defenders Lisa Boughton and Brendan Max – said records showed that ShotSpotter actually initially picked up what sounded like a firework a mile away, and this was later reclassified by ShotSpotter staff to be a gunshot at the intersection where and when Williams was seen on camera. ShotSpotter strongly insisted it had not improperly altered any data to favor the police's case, and said that regardless of the initial real-time alert, its evidence of the gunshot was the result of follow-up forensic analysis, which was submitted to the courts. "Strongly insisted" links to: https://www.theregister.com/2021/08/02/nvidia_cuda_openai/ https://www.theregister.com/2021/08/02/nvidia_cuda_openai/ > One of the pieces of evidence against Williams claims ShotSpotter's sensors in Chicago identified gunfire where surveillance cameras had seen Williams stop his car by a south-side Chicago block, right when and where the cops said Herring was shot. > However, Williams’ lawyer submitted paperwork [PDF] claiming ShotSpotter actually detected a firework a mile away from that location, and that ShotSpotter later reclassified the bang as a gunshot and the location as being where Williams was seen on camera, Vice first reported. > Williams' lawyer demanded the court hold an inquiry into the ShotSpotter evidence, and the prosecutors simply withdrew it. > ShotSpotter responded by denying at length it improperly altered any data or evidence, and hit back at any suggestion it had done so to help the police make a case. It said its software generates real-time alerts automatically, and staff later analyze the microphone readings to submit forensic reports for the courts, and these final reports can therefore differ from the initial alerts. > "The idea that ShotSpotter 'alters' or 'fabricates' evidence in any way is an outrageous lie and would be a criminal offense," it said in a statement. "We follow the facts and data for our forensic analysis. Period." > Update: The case against Williams was dismissed by the judge at the request of the prosecution, which admitted it now had insufficient evidence. Williams had spent the best part of a year in jail awaiting trial.
  - merrywhether 5y ago
    Wow, we need a civil rights lawyer to represent Williams in a wrongful imprisonment suit against the city, if not also a civil suit against ShotSpotter (or maybe even a criminal complaint for knowingly providing false testimony?). People shouldn’t be able to just dust their hands of this.
- coldtea 5y ago
  >So the AI didn’t even make the call. The staff did, manually. (a) The staff of the company who makes the AI, which would have an incentive to make the AI's original judgement appear accurate? (b) The staff of a company paid by the state/police, which aims to please its customers? (c) Was the staff trained/experts in recognizing such sounds, or were they random AI company employees, which just made a judgement call? If so, was this call presented as AI-supported to the trial, and was it sold as anything beyond "some devs, no gun experts, mind you, heard this recording and said this must be a gunshot".
  - dylan604 5y ago
    It could come at the behest of police as in "it would be really helpful to our case if there was ShotSpotter evidence to support our cased". It could come from the fact that ShotSpotter is losing customers, and willing to do anything to make the product "feel" more valuable to the police to ensure cash cows keep producing. The modern criminal investigation has so much negative baggage attached, from historical shenianigans, that it's easy to find ways to question the prosecution's case.
- emodendroket 5y ago
  If manual reclassification is routine then what does that say about the AI?
  - CrazyStat 5y ago
    That it doesn't give the answers the customers (police and prosecutors) want. Which is mostly unrelated to whether or not it's accurate.
    - emodendroket 5y ago
      I think it’s related. If it is good enough to put someone in jail there shouldn’t be any kind of need for manual overrides at all.
      - CrazyStat 5y ago
        "Good" in what sense? Police and prosecutors don't necessarily care about their evidence being good in the sense of accurate. It just needs to convince a jury. There is a very, very, very long history of police and prosecutors using bad (in the sense of accuracy) evidence to put people in jail.
        emodendroket 5y ago
        I’m making a normative claim. Obviously it does not describe reality.
- kbenson 5y ago
  There was a submission here a few weeks back about ShitSpotter and how they seem willing to put in overrides at request of the police without necessarily having a basis for it. If that's what happened in this circumstances it would be, police check ShoySpotter for more evidence, ShotSpotter hasn't automatically marked anything, police request manual review for gunshot they are sure must be there, ShotSpotter manual review process comes up with something and marks it as a shot with a human override. There's a lot of places bias, or outright corruption, could cause problems in that process, if it's what happened here.
- mola 5y ago
  It's the AI companies to blame, always. Obviously the algorithm is without blame, it's not a blamelable entity.
- FireBeyond 5y ago
  What almost invariably happens in cases like this or the cell phone surveillance boxes (Stingrays) is that if the underlying technology or behavior is challenged or questioned strongly enough, the evidence introduced by them, if not the case itself, will be withdrawn, in order to try to avoid any negative precedent against the device or technology being set.
- chacham15 5y ago
  This is missing a key crucial insight: the overall number of evaluated cases increases when AI is helping meaning that borderline cases which may have been ignored prior are now being evaluated. Said another way, with AI handling the obvious cases, the number of borderline cases being evaluated by humans increases. With humans having large amounts of inaccuracy, borderline cases before which may have been ignored due to not having manpower to evaluate are now being evaluated. This can and does lead to many more erroneous accusations despite the fact that all the borderline evaluations are being handled by humans. Another factor here is that simply evaluating more data is also likely to lead to more erroneous evaluations since random chance has a much larger impact.
- smsm42 5y ago
  But this is so convenient! "Your honor, this is the AI algorithm that said it was a gunshot, AI can't lie and is based on pure Science!" Yeah, later it turns out people made that decision because police asked them to do it, but the defense may not know it.
- jrochkind1 5y ago
  It sounds like they reclassified it after the police said "Gee, are you sure you don't have a gunshot? in a slightly different location?" I wonder how much of "AI" introduced in court (or for a warrant) involves some element of individual human attention though... They just call it "AI" without qualification until they get caught (if even then). Would it be better though if it had been purely the output of a machine learning algorithm? (Which after all is not infallible). In that case, the guy probably would have been convicted on that alone...
yawaworht1978 5y ago
Well that's still a failure on both the software programmed by humans causing a false positive and then doubled down by a wrong human review, based on a false alert. All involved stakeholders have failed.
- nend 5y ago
  How is this a failure by software? The software categorized the sound as a fire work.
  - indymike 5y ago
    > How is this a failure by software? The outcome - a guy being charged with a crime using bogus evidence is the story. The software may have been right, but the people who operate the software didn't trust it. Regardless, the story is trying to point out we might not be ready to put our faith in AI for criminal prosecutions at this time.
    - MichaelGroves 5y ago
      If shotspotter had not existed, I think this man would have still been charged with the murder. It seems beyond dispute that the victim was shot in the car of the accused. The case against him was seemingly lost for two reasons: shotspotter's fuckery, and the police not bothering to collect other evidence in a timely matter. It should not have been hard for the police to determine whether there was a gunshot fired in the car, or whether the passenger was struck by a bullet fired from outside the car. There was really no reason for shotspotter to play a role in this case, it could have been cut and dry but they went and muddled the whole thing up with their janky tech.
    - jjk166 5y ago
      If you use a hammer to drive a nail, and the nail goes in exactly as intended, but then the carpenter rips out the nail, how is that a failure of the hammer?
  - notahacker 5y ago
tyingq 5y ago
It's interesting to me that there's no working process to introduce/accept new types of science in court. Polygraphs aren't allowed (though the police still find ways to use it to bully people). So, there's some process. But you still see stories about subsets of fiber, hair, arson, fingerprint, dna, etc, science, that shouldn't have been brought to court.
- jes 5y ago
  The legal system will, over time, also have to accept and adapt to the idea that free will is an illusion. (See “neuro-law” for more information on this.)
  - matt123456789 5y ago
    Unless you are aware of any breakthroughs in fundamental physics or neurology which are being kept from the rest of us, “free will is an illusion” is not a very useful theory when it comes to public policy.
  - amanaplanacanal 5y ago
    The idea that the people who run the legal system have the free will to change it is also an illusion.
  - abrichr 5y ago
    Link: https://en.wikipedia.org/wiki/Neurolaw https://en.wikipedia.org/wiki/Neurolaw > The rapid growth of functional magnetic resonance imaging (fMRI) research has led to new insights on neuroanatomical structure and function, which has led to a greater understanding of human behavior and cognition. As a response, there has been an emergence of questions regarding how these findings can be applied to criminology and legal processes. I hope they have a more solid foundation of evidence than fMRI. Eg from https://www.fastcompany.com/90520750/duke-university-researchers-say-every-brain-activity-study-youve-ever-read-is-wrong https://www.fastcompany.com/90520750/duke-university-researc... : > The researchers reexamined 56 peer-reviewed, published papers that conducted 90 fMRI experiments, some by leaders in the field, and also looked at the results of so-called “test/retest” fMRIs, where 65 subjects were asked to do the same tasks months apart. They found that of seven measures of brain function, none had consistent readings. Also https://www.pnas.org/content/113/28/7900 https://www.pnas.org/content/113/28/7900 : > Functional MRI (fMRI) is 25 years old, yet surprisingly its most common statistical methods have not been validated using real data. Here, we used resting-state fMRI data from 499 healthy controls to conduct 3 million task group analyses. Using this null data with different experimental designs, we estimate the incidence of significant results. In theory, we should find 5% false positives (for a significance threshold of 5%), but instead we found that the most common software packages for fMRI analysis (SPM, FSL, AFNI) can result in false-positive rates of up to 70%. These results question the validity of a number of fMRI studies and may have a large impact on the interpretation of weakly significant neuroimaging results.
Crontab 5y ago
Coming next year: "A man spent a year in jail on child pornography charge that hinged on disputed Apple CSAM evidence".
- csydas 5y ago
  I know you're being coy to slam on Apple, but this already happens in the uS __without__ Apple's (imo misguided) strategy and it happens frequently. Prosecution over thumbnails, file hash matches with tenuous connections to individuals, Crime-Tech has given a new position for law enforcement to craft fanciful narratives to support prosecutions of just about anyone. I'm not specifically citing or inferring there have been *targeted* instances of this, but it happens very often towards US citizens already. Stingrays, Cellebrite, Forensic "science", ShotSpotter, it's absolute chaos and anyone can be a victim here. My goal here is not anything about Apple and its CSAM strategy, but to make it very clear that the imagined scenario happens now and happens frequently.
  - ailef 5y ago
    I'd be interested in reading about some of these real cases if you have any links handy.
- scoopertrooper 5y ago
  For that to happen: they'd have to be matched 30 times, the Apple employees would have to mistakenly manually confirm the match 30 times, and the defence would fail to request to see the underlying images that matched the hashes during discovery. It seems... far fetched.
  - the-dude 5y ago
    For now.
    - scoopertrooper 5y ago
      When your slippery slope includes the abolition of pre-trial discovery, it's a very slippery slope indeed.
JackFr 5y ago
“It's said that in May last year, Williams was driving through Chicago one night hoping to buy some cigarettes.” Ah yes, I remember hearing that a lot growing up. Editors, do your job!
- jwilk 5y ago
  What do you mean?
  - meowster 5y ago
    I think OP meant that "driving through Chicago" was pretty large/vague area for someone just trying to buy cigarettes. It's like saying I was driving through the United States to go to my local corner store. I wouldn't have thought about the poor writing if OP didn't mention it, but it is still a bit pedantic in my opinion.
    - omnicognate 5y ago
      More likely "it's said" - a claim that some unspecified people said something, expressed using a phrase people often interpret to mean "it is generally believed that..." or "there is a saying that goes...". GP sarcastically said sure "it's said" that x occurred, I heard it all the time as I was growing up. It's not good journalism. If someone said something, say who said it. If it's an anonymous source say so. "It's said..." in an article like this means "I vaguely think this has been said somewhere but I can't be bothered to check or quote my sources".
    - meowster 5y ago
      Thank you everyone (including OP) who clarified that it was "it's said".
  - LeonB 5y ago
    “It’s said” — by whom?
  - aaaaaaaaaaab 5y ago
    Weasel words: “it’s said”
deleted 5y ago
[deleted]
tluyben2 5y ago
Not really relevant to this article, but relevant to the title: if AI keeps banning accounts for reasons that seems nonsense to humans, how will we protect people from getting flagged and locked up 'just in case' by the machine? AI is just not advanced enough to jail people, by a very large margin. And yet we will see it happen.
- MeinBlutIstBlau 5y ago
  The scariest part is how quickly policing agencies are to embrace this tech as well but the government as a whole is slow to change anything else. Why is it that the state can be proactive about jailing me but not as much when it comes to quickly resolving cases and protecting my rights?
rvba 5y ago
Wow there are microphones used to detect gunshots in Chicago? I remember seeing such technology in a relatively obscure cyberpunk comic book when I was a kid, but I never thought this would be legal. Do the microphones record conversations too? Can the government spy on its own citizens? I guess some secret court ( https://en.m.wikipedia.org/wiki/United_States_Foreign_Intelligence_Surveillance_Court https://en.m.wikipedia.org/wiki/United_States_Foreign_Intell... ) allowed it?
- bleachedsleet 5y ago
  Oh boy, wait until you learn it’s not just Chicago where they deploy this crap [1] [1] https://www.shotspotter.com/cities/ https://www.shotspotter.com/cities/
- LinuxBender 5y ago
  There are LED street lights in San Diego California that do just that. They have cameras and microphones. The microphones are used for Machine Learning of what people say. [1] There are a handful of other cities deploying these. It is a clever way to deploy CCTV much like what the UK has but without drawing much attention to it. [1] - https://www.smartcitiesdive.com/news/san-diego-faces-lawsuit-over-streetlight-data-collection/569415/ https://www.smartcitiesdive.com/news/san-diego-faces-lawsuit...
leephillips 5y ago
This year in jail was awaiting the “speedy trial” that is supposed to be guaranteed by the US Constitution. I think that in itself is as much of an issue as the quality of the evidence. How can the government justify taking year away from the life of a man who has not been convicted of anything?
- stelcodes 5y ago
  A year? Curtis Flowers spent over 20 years on Death Row without being convicted of anything. A couple years ago he was set free! Why? Because of a true crime podcast called In The Dark. Seriously. Listen to season 2 and your mind will be blown. The “justice” system in this country is horrific. https://en.wikipedia.org/wiki/Curtis_Flowers?wprov=sfti1 https://en.wikipedia.org/wiki/Curtis_Flowers?wprov=sfti1
  - leephillips 5y ago
    I agree with your distress at the state of the justice system, but it doesn’t make sense to say that someone was on death row without having been convicted. The article you link to explains that his convictions were overturned. Although I would never rely on information in a Wikipedia article, you did link to it, so presumably you find it credible.
    - stelcodes 5y ago
      Every time he was “convicted”, the prosecution was found to be in blatant violation of laws on appeal. They intimidated witnesses, kept black people off the jury, and more. So no, he was never rightfully convicted! The prosecution broke so many laws, way more than Curtis Flowers ever did! And they just did it over and over again for 20 years. None of them have faced any consequences. They effectively imprisoned an innocent man for 20 years because they didn’t like him. And big surprise, Curtis is black.
      - ClumsyPilot 5y ago
        What you are describing is different from convicting someone withoht trial, it's way worse.
aaron695 5y ago
Do we get the AI doesn't detect location? This is just high school maths, pretty obvious I guess. The "AI" is just used to see if it's a gunshot for early detection. Obviously anyone can listen in later to the many recording (min 3) to see what they think about the noise. So the gun shot was in a car, so the mathematics fails. This is why it was withdrawn. So if you were blaming AI it's important you blame mathematics instead now you are better informed. Because the only thing I hate more than the fake AI industry is people with an inconsistent stance.
tomxor 5y ago
> “I kept trying to figure out, how can they get away with using the technology like that against me,” I have a bad feeling this will be a theme over the next decade as government and law enforcement around the world starts to integrate "AI" (or rather call it what it is: statistical models + mass surveilence), without enough scepticism built into their process. Some of us know better, but it looks like society and government will still have to learn the hard way that these tools are not a highlight pen, they are a very crude spray can, and even then they will miss spots.
LatteLazy 5y ago
This seems like the definition of circumstantial evidence doesn't it? AI, human, accourate or not, he shouldn't have been in jail without something more.
emodendroket 5y ago
I’m not a Luddite by any means but it is time to be more honest about the limitations of AI.
- MeinBlutIstBlau 5y ago
  The only time I hear about people who talk about AI are average joes, ignorant journalists, generic business analysts, and upper management that don't know anything about programming. I wish I could point to these people and say "look at video game AI, we can't even make those without it cheating or following a routine to simulate human playstyles. What makes you think we can do that in real life without doing the same?"
  - emodendroket 5y ago
    Video games are a bad example because it’s pretty feasible to make an AI that would win every time.
    - MeinBlutIstBlau 5y ago
      They are a perfect example because of that. They are in controlled environments. Life is the testing ground of everything. To think we can trust a computer to have reasoning skills in our environment ethically is absurd and a total affront to freedom.
    - shadilay 5y ago
      Only in very simple games. There's a reason that there's not more AI multiplayer grinding bots.
setr 5y ago
It seems to me jail time without verdict is the real problem in this scenario — it’s not clear to me how that’s not “guilty until proven innocent”
- DubiousPusher 5y ago
  Precisely. This happens to people all the time regardless of the technology involved. Being suspected of a crime should never ruin your life. At most it should pause your life. House arrest with deferral of your debts and obligations and mandatory reinstatement of your job if you are exhonorated or the charges are removed seems like what you would need at a minimum to live up to "innocent until found guilty".
  - qayxc 5y ago
    That's exactly what judges decide to do if certain criteria are met, though. The person in this case, however, had a criminal history including attempted homicide so the judge decided to opt for detention while awaiting trial instead. Sounds like perfectly normal procedure to me and happens other countries as well.
- MeinBlutIstBlau 5y ago
  In Germany and Austria you get compensated for your time incarcerated if found innocent. So as to basically "fine" the government for not having speedy trials.
  - bagacrap 5y ago
    that sounds fair except not great that it's just penalizing the taxpayer, not anyone in government
    - MeinBlutIstBlau 5y ago
      We need to quit thinking about "penalizing the taxpayer" type of mentality. It's the governments income. You don't get to act like a taxpayer for all the gas you buy or interest you pay in, but suddenly when money funnels to the government your money and how it's spent matters. The government is just another corporate entity. Not like you get the money they tax back anyway.
StreamBright 5y ago
The US has the most troublesome track record of law enforcement abuse of "evidence". https://innocenceproject.org/what-is-bite-mark-evidence-forensic-science/ https://innocenceproject.org/what-is-bite-mark-evidence-fore... https://bostonreview.net/books-ideas/nathan-robinson-forensic-pseudoscience-criminal-justice https://bostonreview.net/books-ideas/nathan-robinson-forensi... https://archive.nytimes.com/www.nytimes.com/books/first/k/kelly-evidence.html https://archive.nytimes.com/www.nytimes.com/books/first/k/ke...
Borrible 5y ago
Fake AI? Who would have thought? Shocking, shocking I tell you. https://venturebeat.com/2021/04/05/government-audit-of-ai-with-ties-to-white-supremacy-finds-no-ai/ https://venturebeat.com/2021/04/05/government-audit-of-ai-wi...
siliconc0w 5y ago
Wow - any software system that is used to submit evidence to the court should have a preferably cryptographically secure chain of custody of all artifacts - including the raw recorded audio and the code used to process it. Any manual 'post processing' should be tracked down to a person, a date, and a verifiable reason for making changes to the model/decision. For 'black box' models an independent organization should be able to test them to confirm stated accuracy/precision. I don't think the answer is to do away with these systems but as we yield more and more authority to machines regulation will need to catch up so they aren't just used a fig leaf for corroboratory-evidence-as-a-service.
bryan0 5y ago
> Soon after another vehicle pulled up alongside, and someone in a passenger seat took out a gun and shot Herring in the head, Williams told police. I dont understand what Shotspotter has to do with this case at all. The question the case seems to hinge on is “who fired the gun”, not “was a gun fired at this location”. What am I missing?
- dylan604 5y ago
  >What am I missing? The fact that some of the evidence the police used to say who did what where was manufactured by the company.
  - bryan0 5y ago
    The headline is: > A man spent a year in jail on a murder charge that hinged on disputed AI evidence. But after the reading the article, the evidence seemed irrelevant? Maybe the headline should have been: > AI evidence was manually reclassified after human review in murder case
    - DubiousPusher 5y ago
      Incorrect > Prosecutors said ShotSpotter picked up a gunshot sound where Williams was seen on surveillance camera footage in his car, putting it all forward as proof that Williams shot Herring right there and then. The police used the altered location of the shot to imply that the defendant's story about what happened wasn't true. Which was that another person in another var at a different location did the shooting. They used the altered location data alongside video evidence to imply the defendant and the victim were alone at the shot location and therefore the defendant was the only possible suspect.
      - bryan0 5y ago
        > The police used the altered location of the shot to imply that the defendant's story about what happened wasn't true. Which was that another person in another var at a different location did the shooting. Ok that would make more sense, but I definitely did not take that away from the article. I’m sure there are better sources on this case though.
arthur_sav 5y ago
Anyone else surprised that such a system is running across cities?
kevin_thibedeau 5y ago
I get that their system may sometimes need manual classification of signals but how is it possible to change the triangulated location? That is the core part of the technology and doesn't require ML to implement. Seems like it's all just Theranos style vapor.
- c_weighted 5y ago
  Locating on an echo at one more receivers rather than a perhaps quieter more direct path. Putting the wrong impulses together when computing a location. Computing the wrong start time for one or more impulses. If you want to read up on all the ways this kind of thing can go wrong, the technical term here is "source localization."
  - kevin_thibedeau 5y ago
    The military has no problem detecting gunfire direction and estimating range from a single microphone array on a mast. Shotspotter is a distributed network of multiple sensors across a wide area. If they can't make it work after 15+ years then they are hopeless.
    - c_weighted 5y ago
      Boomerang is colocated with the target. Multipath is less likely to be an issue because there is likely to be line of sight to the source. Selecting the right impulses is easier because the spacing of the microphones provides some tight timing constraints. You’re not likely to see inconsistencies in timing the impulses lead to location error because any effects of the environment are likely to be consistent across the pulses. Finally, the location accuracy requirements are far different. ShotSpotter needs to put a dot on a map accurate enough to find shell casings or a victim in a wide geographic area. Boomerang just needs to give a good bearing from the receivers to return fire. The range doesn’t have to be all that precise, and the mathematics suggests it isn’t. These are two very different approaches to solve different problems.
backtoyoujim 5y ago
>“The problem? When we made it possible for anyone to generate art with artificial intelligence, barely anyone used it to make actual art. Instead, our AI model was forced to make videos for random inputs, trolling queries, and NSFW intents.” I am struggling with the use of the word "forced"; the researchers put their software out there to be used by anyone with an internet connection. The researchers created that dynamic. And how are they not seeing that AI created no-humans-used-to-make-this-porn could be a net good.
Vaslo 5y ago
We have this system in our city. Can’t speak to the accuracy and whether or not it should be admitted evidence. It has in some cases revealed the approximate locations of victims who have been shot where they potentially could have died without the AI heads up as to where they were.
TeeMassive 5y ago
Prosecution based on evidence provided by closed source and closed hardware based devices should never have been admitted in courts of laws in the first place.

372 points by LinuxBender 5y ago

This is about ShotSpotter. https://hn.algolia.com/?q=ShotSpotter https://hn.algolia.com/?q=ShotSpotter

The article says that employees of the AI company (ShotSpotter) manually reviewed and classified the sounds as gunshots: > records showed that ShotSpotter actually initially picked up what sounded like a firework a mile away, and this was later reclassified by ShotSpotter staff to be a gunshot at the intersection where and when Williams was seen on camera. So the AI didn’t even make the call. The staff did, manually. I assume that means the actual audio is available and entered into evidence? If humans are making the call then blaming AI seems like a stretch. That’s almost like blaming the motion detection algorithms for triggering video recordings that were later reviewed by humans. It’s still humans reviewing the recordings and making decisions.

jtbayly 5y ago
It may not be right to blame AI, but it's certainly the case that they were presenting it as "the system says there was a shot fired here." The bigger point is that the "system" can't be trusted. And the fact that they need to manually classify things is proof of that fact. (It's way more complicated than just an audio recording. It's recordings from numerous locations, attempting to triangulate the original location, without being confused by echoes and other things.)
- PragmaticPulp 5y ago
  > It may not be right to blame AI, but it's certainly the case that they were presenting it as "the system says there was a shot fired here." I think a crucial distinction is to clarify who “they” is: Who was making the claim? From the article, it seems that the prosecutors may have been the ones exaggerating the evidence: > Prosecutors said ShotSpotter picked up a gunshot sound where Williams was seen on surveillance camera footage in his car, putting it all forward as proof that Williams shot Herring right there and then. Naturally the defense wanted to see the actual evidence, so that’s what they got down to. I know everyone wants to vilify AI, but I think the actual bad actor in this article might be the prosecutor trying to misrepresent the evidence.
  - Aerroon 5y ago
    >I think a crucial distinction is to clarify who “they” is: Who was making the claim? From the article, it seems that the prosecutors may have been the ones exaggerating the evidence: The prosecutors are the "they". The further removed you get from people who know the 'AI' the more the validity of the evidence provided by the AI relies merely on what others have said. The people working on the system know that it can be faulty. Other software developers have an idea that it could be faulty. Prosecutors are told that it's alright. They turn around and argue it in a court of law. And the further you get the less people have any idea that the system is faulty. Down the line people will go "the machine says so, therefore it must be so".
    - ClumsyPilot 5y ago
      >Down the line people will go "the machine says so, therefore it must be so". Thats the real feature of AI, devision laundering: a flawed decision made by someone for their own gain now appears impartial and trustworthy, and come wothout accountability
  - jtbayly 5y ago
    Sure. But the point is that they misrepresent the evidence by pointing to the system as trustworthy. Even without tampering with the evidence the system isn’t trustworthy beyond a reasonable shadow of a doubt.
stefan_ 5y ago
That's the point? These systems invariably turn out to not be any sort of AI at all, but sales people ready to make a call any way their customer (police) wants it made. Later it's presented as AI to a jury. It's basically the same scam as lots of forensics, junk science to get a desired result.
- giantg2 5y ago
  I generally agree, but wouldn't necessarily say a lot of forensics is junk science. The science is generally sound (procedures and theory). Usually it's the prosecutors that present the forensics in a way that misrepresents what the results actually mean. Or lab techs make mistakes or perform misconduct (How to Fix a Drug Scandal on Netflix is interesting).
  - wyager 5y ago
    Fingerprinting is junk science. Polygraphs are junk science. Tons of things that have been or are admissible as evidence are junk science.
    - giantg2 5y ago
      I wouldn't call polygraphs forensics. Most states do not allow them as evidence. Do you have a link for the fingerprinting claim?
      - monetus 5y ago
        (Not who you responded to) https://www.crammlawfirm.com/fingerprint-match-jails-innocent-man/ https://www.crammlawfirm.com/fingerprint-match-jails-innocen... This isn't the case I originally recalled, and doesn't say how many points were matched, but I remember a 5 point match being used to convict a person who was later proven innocent.
        newswasboring 5y ago
        That shows its flawed, not junk.
helsinkiandrew 5y ago
It's scary that prosecutors thought that the ShotSpotter evidence is suitable for a court. It's amazing technology that can help police get to a crime scene quickly but is far too easy to spoof or get the wrong answer due to reflections and other sounds etc. Hopefully cases like this will eventually stop the system being used in court.
- SamoyedFurFluff 5y ago
  I mean, drug dogs are still a thing even though we know they tend to respond to owners desires regardless of if there’s drugs present.
  - yawaworht1978 5y ago
    Wow i have never heard of that. Curious to see a link/article on the issue, if you can provide.
    - DaftDank 5y ago
      This is a decent summary of court cases related to the dogs: https://www.nlrg.com/criminal-law-legal-research/bid/84670/CRIMINAL-LAW-UPDATE-Reliability-of-Narcotics-Dogs-to-Be-Revisited-by-U-S-Supreme-Court https://www.nlrg.com/criminal-law-legal-research/bid/84670/C... I think this part is pretty key, and has always been a big open question in my mind: "The court further recognized a potential problem in assessing reliability related to handler cuing, as even well-trained dogs could respond to subconscious cues from their handlers. For this reason, the court determined that a critical factor in determining reliability is the record of false positive alerts made by the dog." The subconscious cues, or even deliberate cues that only the handler and dog know, could allow an officer with ill intentions to make some kind of signal only known to him and the dog, which causes the dog to make an "alert", giving the cop probable cause.
      - leephillips 5y ago
        Some departments deploy dogs that alert on all vehicles when they want an excuse to search. In at least one department, this was referred to as “probable cause on four legs”. If judges continue to allow this quality of evidence, the police will continue to abuse the legal environment that creates.
nerdponx 5y ago
> Crucially, Williams' lawyers – public defenders Lisa Boughton and Brendan Max – said records showed that ShotSpotter actually initially picked up what sounded like a firework a mile away, and this was later reclassified by ShotSpotter staff to be a gunshot at the intersection where and when Williams was seen on camera. ShotSpotter strongly insisted it had not improperly altered any data to favor the police's case, and said that regardless of the initial real-time alert, its evidence of the gunshot was the result of follow-up forensic analysis, which was submitted to the courts. "Strongly insisted" links to: https://www.theregister.com/2021/08/02/nvidia_cuda_openai/ https://www.theregister.com/2021/08/02/nvidia_cuda_openai/ > One of the pieces of evidence against Williams claims ShotSpotter's sensors in Chicago identified gunfire where surveillance cameras had seen Williams stop his car by a south-side Chicago block, right when and where the cops said Herring was shot. > However, Williams’ lawyer submitted paperwork [PDF] claiming ShotSpotter actually detected a firework a mile away from that location, and that ShotSpotter later reclassified the bang as a gunshot and the location as being where Williams was seen on camera, Vice first reported. > Williams' lawyer demanded the court hold an inquiry into the ShotSpotter evidence, and the prosecutors simply withdrew it. > ShotSpotter responded by denying at length it improperly altered any data or evidence, and hit back at any suggestion it had done so to help the police make a case. It said its software generates real-time alerts automatically, and staff later analyze the microphone readings to submit forensic reports for the courts, and these final reports can therefore differ from the initial alerts. > "The idea that ShotSpotter 'alters' or 'fabricates' evidence in any way is an outrageous lie and would be a criminal offense," it said in a statement. "We follow the facts and data for our forensic analysis. Period." > Update: The case against Williams was dismissed by the judge at the request of the prosecution, which admitted it now had insufficient evidence. Williams had spent the best part of a year in jail awaiting trial.
- merrywhether 5y ago
  Wow, we need a civil rights lawyer to represent Williams in a wrongful imprisonment suit against the city, if not also a civil suit against ShotSpotter (or maybe even a criminal complaint for knowingly providing false testimony?). People shouldn’t be able to just dust their hands of this.
coldtea 5y ago
>So the AI didn’t even make the call. The staff did, manually. (a) The staff of the company who makes the AI, which would have an incentive to make the AI's original judgement appear accurate? (b) The staff of a company paid by the state/police, which aims to please its customers? (c) Was the staff trained/experts in recognizing such sounds, or were they random AI company employees, which just made a judgement call? If so, was this call presented as AI-supported to the trial, and was it sold as anything beyond "some devs, no gun experts, mind you, heard this recording and said this must be a gunshot".
- dylan604 5y ago
  It could come at the behest of police as in "it would be really helpful to our case if there was ShotSpotter evidence to support our cased". It could come from the fact that ShotSpotter is losing customers, and willing to do anything to make the product "feel" more valuable to the police to ensure cash cows keep producing. The modern criminal investigation has so much negative baggage attached, from historical shenianigans, that it's easy to find ways to question the prosecution's case.
emodendroket 5y ago
If manual reclassification is routine then what does that say about the AI?
- CrazyStat 5y ago
  That it doesn't give the answers the customers (police and prosecutors) want. Which is mostly unrelated to whether or not it's accurate.
  - emodendroket 5y ago
    I think it’s related. If it is good enough to put someone in jail there shouldn’t be any kind of need for manual overrides at all.
    - CrazyStat 5y ago
      "Good" in what sense? Police and prosecutors don't necessarily care about their evidence being good in the sense of accurate. It just needs to convince a jury. There is a very, very, very long history of police and prosecutors using bad (in the sense of accuracy) evidence to put people in jail.
      - emodendroket 5y ago
        I’m making a normative claim. Obviously it does not describe reality.
kbenson 5y ago
There was a submission here a few weeks back about ShitSpotter and how they seem willing to put in overrides at request of the police without necessarily having a basis for it. If that's what happened in this circumstances it would be, police check ShoySpotter for more evidence, ShotSpotter hasn't automatically marked anything, police request manual review for gunshot they are sure must be there, ShotSpotter manual review process comes up with something and marks it as a shot with a human override. There's a lot of places bias, or outright corruption, could cause problems in that process, if it's what happened here.
mola 5y ago
It's the AI companies to blame, always. Obviously the algorithm is without blame, it's not a blamelable entity.
FireBeyond 5y ago
What almost invariably happens in cases like this or the cell phone surveillance boxes (Stingrays) is that if the underlying technology or behavior is challenged or questioned strongly enough, the evidence introduced by them, if not the case itself, will be withdrawn, in order to try to avoid any negative precedent against the device or technology being set.
chacham15 5y ago
This is missing a key crucial insight: the overall number of evaluated cases increases when AI is helping meaning that borderline cases which may have been ignored prior are now being evaluated. Said another way, with AI handling the obvious cases, the number of borderline cases being evaluated by humans increases. With humans having large amounts of inaccuracy, borderline cases before which may have been ignored due to not having manpower to evaluate are now being evaluated. This can and does lead to many more erroneous accusations despite the fact that all the borderline evaluations are being handled by humans. Another factor here is that simply evaluating more data is also likely to lead to more erroneous evaluations since random chance has a much larger impact.
smsm42 5y ago
But this is so convenient! "Your honor, this is the AI algorithm that said it was a gunshot, AI can't lie and is based on pure Science!" Yeah, later it turns out people made that decision because police asked them to do it, but the defense may not know it.
jrochkind1 5y ago
It sounds like they reclassified it after the police said "Gee, are you sure you don't have a gunshot? in a slightly different location?" I wonder how much of "AI" introduced in court (or for a warrant) involves some element of individual human attention though... They just call it "AI" without qualification until they get caught (if even then). Would it be better though if it had been purely the output of a machine learning algorithm? (Which after all is not infallible). In that case, the guy probably would have been convicted on that alone...

yawaworht1978 5y ago

Well that's still a failure on both the software programmed by humans causing a false positive and then doubled down by a wrong human review, based on a false alert. All involved stakeholders have failed.

nend 5y ago
How is this a failure by software? The software categorized the sound as a fire work.
- indymike 5y ago
  > How is this a failure by software? The outcome - a guy being charged with a crime using bogus evidence is the story. The software may have been right, but the people who operate the software didn't trust it. Regardless, the story is trying to point out we might not be ready to put our faith in AI for criminal prosecutions at this time.
  - MichaelGroves 5y ago
    If shotspotter had not existed, I think this man would have still been charged with the murder. It seems beyond dispute that the victim was shot in the car of the accused. The case against him was seemingly lost for two reasons: shotspotter's fuckery, and the police not bothering to collect other evidence in a timely matter. It should not have been hard for the police to determine whether there was a gunshot fired in the car, or whether the passenger was struck by a bullet fired from outside the car. There was really no reason for shotspotter to play a role in this case, it could have been cut and dry but they went and muddled the whole thing up with their janky tech.
  - jjk166 5y ago
    If you use a hammer to drive a nail, and the nail goes in exactly as intended, but then the carpenter rips out the nail, how is that a failure of the hammer?
- notahacker 5y ago

tyingq 5y ago

It's interesting to me that there's no working process to introduce/accept new types of science in court. Polygraphs aren't allowed (though the police still find ways to use it to bully people). So, there's some process. But you still see stories about subsets of fiber, hair, arson, fingerprint, dna, etc, science, that shouldn't have been brought to court.

jes 5y ago
The legal system will, over time, also have to accept and adapt to the idea that free will is an illusion. (See “neuro-law” for more information on this.)
- matt123456789 5y ago
  Unless you are aware of any breakthroughs in fundamental physics or neurology which are being kept from the rest of us, “free will is an illusion” is not a very useful theory when it comes to public policy.
- amanaplanacanal 5y ago
  The idea that the people who run the legal system have the free will to change it is also an illusion.
- abrichr 5y ago
  Link: https://en.wikipedia.org/wiki/Neurolaw https://en.wikipedia.org/wiki/Neurolaw > The rapid growth of functional magnetic resonance imaging (fMRI) research has led to new insights on neuroanatomical structure and function, which has led to a greater understanding of human behavior and cognition. As a response, there has been an emergence of questions regarding how these findings can be applied to criminology and legal processes. I hope they have a more solid foundation of evidence than fMRI. Eg from https://www.fastcompany.com/90520750/duke-university-researchers-say-every-brain-activity-study-youve-ever-read-is-wrong https://www.fastcompany.com/90520750/duke-university-researc... : > The researchers reexamined 56 peer-reviewed, published papers that conducted 90 fMRI experiments, some by leaders in the field, and also looked at the results of so-called “test/retest” fMRIs, where 65 subjects were asked to do the same tasks months apart. They found that of seven measures of brain function, none had consistent readings. Also https://www.pnas.org/content/113/28/7900 https://www.pnas.org/content/113/28/7900 : > Functional MRI (fMRI) is 25 years old, yet surprisingly its most common statistical methods have not been validated using real data. Here, we used resting-state fMRI data from 499 healthy controls to conduct 3 million task group analyses. Using this null data with different experimental designs, we estimate the incidence of significant results. In theory, we should find 5% false positives (for a significance threshold of 5%), but instead we found that the most common software packages for fMRI analysis (SPM, FSL, AFNI) can result in false-positive rates of up to 70%. These results question the validity of a number of fMRI studies and may have a large impact on the interpretation of weakly significant neuroimaging results.

Crontab 5y ago

Coming next year: "A man spent a year in jail on child pornography charge that hinged on disputed Apple CSAM evidence".

csydas 5y ago
I know you're being coy to slam on Apple, but this already happens in the uS __without__ Apple's (imo misguided) strategy and it happens frequently. Prosecution over thumbnails, file hash matches with tenuous connections to individuals, Crime-Tech has given a new position for law enforcement to craft fanciful narratives to support prosecutions of just about anyone. I'm not specifically citing or inferring there have been *targeted* instances of this, but it happens very often towards US citizens already. Stingrays, Cellebrite, Forensic "science", ShotSpotter, it's absolute chaos and anyone can be a victim here. My goal here is not anything about Apple and its CSAM strategy, but to make it very clear that the imagined scenario happens now and happens frequently.
- ailef 5y ago
  I'd be interested in reading about some of these real cases if you have any links handy.
scoopertrooper 5y ago
For that to happen: they'd have to be matched 30 times, the Apple employees would have to mistakenly manually confirm the match 30 times, and the defence would fail to request to see the underlying images that matched the hashes during discovery. It seems... far fetched.
- the-dude 5y ago
  For now.
  - scoopertrooper 5y ago
    When your slippery slope includes the abolition of pre-trial discovery, it's a very slippery slope indeed.

JackFr 5y ago

“It's said that in May last year, Williams was driving through Chicago one night hoping to buy some cigarettes.” Ah yes, I remember hearing that a lot growing up. Editors, do your job!

jwilk 5y ago
What do you mean?
- meowster 5y ago
  I think OP meant that "driving through Chicago" was pretty large/vague area for someone just trying to buy cigarettes. It's like saying I was driving through the United States to go to my local corner store. I wouldn't have thought about the poor writing if OP didn't mention it, but it is still a bit pedantic in my opinion.
  - omnicognate 5y ago
    More likely "it's said" - a claim that some unspecified people said something, expressed using a phrase people often interpret to mean "it is generally believed that..." or "there is a saying that goes...". GP sarcastically said sure "it's said" that x occurred, I heard it all the time as I was growing up. It's not good journalism. If someone said something, say who said it. If it's an anonymous source say so. "It's said..." in an article like this means "I vaguely think this has been said somewhere but I can't be bothered to check or quote my sources".
  - meowster 5y ago
    Thank you everyone (including OP) who clarified that it was "it's said".
- LeonB 5y ago
  “It’s said” — by whom?
- aaaaaaaaaaab 5y ago
  Weasel words: “it’s said”

deleted 5y ago

[deleted]

tluyben2 5y ago

Not really relevant to this article, but relevant to the title: if AI keeps banning accounts for reasons that seems nonsense to humans, how will we protect people from getting flagged and locked up 'just in case' by the machine? AI is just not advanced enough to jail people, by a very large margin. And yet we will see it happen.

MeinBlutIstBlau 5y ago
The scariest part is how quickly policing agencies are to embrace this tech as well but the government as a whole is slow to change anything else. Why is it that the state can be proactive about jailing me but not as much when it comes to quickly resolving cases and protecting my rights?

rvba 5y ago

Wow there are microphones used to detect gunshots in Chicago? I remember seeing such technology in a relatively obscure cyberpunk comic book when I was a kid, but I never thought this would be legal. Do the microphones record conversations too? Can the government spy on its own citizens? I guess some secret court ( https://en.m.wikipedia.org/wiki/United_States_Foreign_Intelligence_Surveillance_Court https://en.m.wikipedia.org/wiki/United_States_Foreign_Intell... ) allowed it?

bleachedsleet 5y ago
Oh boy, wait until you learn it’s not just Chicago where they deploy this crap [1] [1] https://www.shotspotter.com/cities/ https://www.shotspotter.com/cities/
LinuxBender 5y ago
There are LED street lights in San Diego California that do just that. They have cameras and microphones. The microphones are used for Machine Learning of what people say. [1] There are a handful of other cities deploying these. It is a clever way to deploy CCTV much like what the UK has but without drawing much attention to it. [1] - https://www.smartcitiesdive.com/news/san-diego-faces-lawsuit-over-streetlight-data-collection/569415/ https://www.smartcitiesdive.com/news/san-diego-faces-lawsuit...

leephillips 5y ago

This year in jail was awaiting the “speedy trial” that is supposed to be guaranteed by the US Constitution. I think that in itself is as much of an issue as the quality of the evidence. How can the government justify taking year away from the life of a man who has not been convicted of anything?

stelcodes 5y ago
A year? Curtis Flowers spent over 20 years on Death Row without being convicted of anything. A couple years ago he was set free! Why? Because of a true crime podcast called In The Dark. Seriously. Listen to season 2 and your mind will be blown. The “justice” system in this country is horrific. https://en.wikipedia.org/wiki/Curtis_Flowers?wprov=sfti1 https://en.wikipedia.org/wiki/Curtis_Flowers?wprov=sfti1
- leephillips 5y ago
  I agree with your distress at the state of the justice system, but it doesn’t make sense to say that someone was on death row without having been convicted. The article you link to explains that his convictions were overturned. Although I would never rely on information in a Wikipedia article, you did link to it, so presumably you find it credible.
  - stelcodes 5y ago
    Every time he was “convicted”, the prosecution was found to be in blatant violation of laws on appeal. They intimidated witnesses, kept black people off the jury, and more. So no, he was never rightfully convicted! The prosecution broke so many laws, way more than Curtis Flowers ever did! And they just did it over and over again for 20 years. None of them have faced any consequences. They effectively imprisoned an innocent man for 20 years because they didn’t like him. And big surprise, Curtis is black.
    - ClumsyPilot 5y ago
      What you are describing is different from convicting someone withoht trial, it's way worse.

aaron695 5y ago

Do we get the AI doesn't detect location? This is just high school maths, pretty obvious I guess. The "AI" is just used to see if it's a gunshot for early detection. Obviously anyone can listen in later to the many recording (min 3) to see what they think about the noise. So the gun shot was in a car, so the mathematics fails. This is why it was withdrawn. So if you were blaming AI it's important you blame mathematics instead now you are better informed. Because the only thing I hate more than the fake AI industry is people with an inconsistent stance.

tomxor 5y ago

> “I kept trying to figure out, how can they get away with using the technology like that against me,” I have a bad feeling this will be a theme over the next decade as government and law enforcement around the world starts to integrate "AI" (or rather call it what it is: statistical models + mass surveilence), without enough scepticism built into their process. Some of us know better, but it looks like society and government will still have to learn the hard way that these tools are not a highlight pen, they are a very crude spray can, and even then they will miss spots.

LatteLazy 5y ago

This seems like the definition of circumstantial evidence doesn't it? AI, human, accourate or not, he shouldn't have been in jail without something more.

emodendroket 5y ago

I’m not a Luddite by any means but it is time to be more honest about the limitations of AI.

MeinBlutIstBlau 5y ago
The only time I hear about people who talk about AI are average joes, ignorant journalists, generic business analysts, and upper management that don't know anything about programming. I wish I could point to these people and say "look at video game AI, we can't even make those without it cheating or following a routine to simulate human playstyles. What makes you think we can do that in real life without doing the same?"
- emodendroket 5y ago
  Video games are a bad example because it’s pretty feasible to make an AI that would win every time.
  - MeinBlutIstBlau 5y ago
    They are a perfect example because of that. They are in controlled environments. Life is the testing ground of everything. To think we can trust a computer to have reasoning skills in our environment ethically is absurd and a total affront to freedom.
  - shadilay 5y ago
    Only in very simple games. There's a reason that there's not more AI multiplayer grinding bots.

setr 5y ago

It seems to me jail time without verdict is the real problem in this scenario — it’s not clear to me how that’s not “guilty until proven innocent”

DubiousPusher 5y ago
Precisely. This happens to people all the time regardless of the technology involved. Being suspected of a crime should never ruin your life. At most it should pause your life. House arrest with deferral of your debts and obligations and mandatory reinstatement of your job if you are exhonorated or the charges are removed seems like what you would need at a minimum to live up to "innocent until found guilty".
- qayxc 5y ago
  That's exactly what judges decide to do if certain criteria are met, though. The person in this case, however, had a criminal history including attempted homicide so the judge decided to opt for detention while awaiting trial instead. Sounds like perfectly normal procedure to me and happens other countries as well.
MeinBlutIstBlau 5y ago
In Germany and Austria you get compensated for your time incarcerated if found innocent. So as to basically "fine" the government for not having speedy trials.
- bagacrap 5y ago
  that sounds fair except not great that it's just penalizing the taxpayer, not anyone in government
  - MeinBlutIstBlau 5y ago
    We need to quit thinking about "penalizing the taxpayer" type of mentality. It's the governments income. You don't get to act like a taxpayer for all the gas you buy or interest you pay in, but suddenly when money funnels to the government your money and how it's spent matters. The government is just another corporate entity. Not like you get the money they tax back anyway.

StreamBright 5y ago

The US has the most troublesome track record of law enforcement abuse of "evidence". https://innocenceproject.org/what-is-bite-mark-evidence-forensic-science/ https://innocenceproject.org/what-is-bite-mark-evidence-fore... https://bostonreview.net/books-ideas/nathan-robinson-forensic-pseudoscience-criminal-justice https://bostonreview.net/books-ideas/nathan-robinson-forensi... https://archive.nytimes.com/www.nytimes.com/books/first/k/kelly-evidence.html https://archive.nytimes.com/www.nytimes.com/books/first/k/ke...

Borrible 5y ago

Fake AI? Who would have thought? Shocking, shocking I tell you. https://venturebeat.com/2021/04/05/government-audit-of-ai-with-ties-to-white-supremacy-finds-no-ai/ https://venturebeat.com/2021/04/05/government-audit-of-ai-wi...

siliconc0w 5y ago

Wow - any software system that is used to submit evidence to the court should have a preferably cryptographically secure chain of custody of all artifacts - including the raw recorded audio and the code used to process it. Any manual 'post processing' should be tracked down to a person, a date, and a verifiable reason for making changes to the model/decision. For 'black box' models an independent organization should be able to test them to confirm stated accuracy/precision. I don't think the answer is to do away with these systems but as we yield more and more authority to machines regulation will need to catch up so they aren't just used a fig leaf for corroboratory-evidence-as-a-service.

bryan0 5y ago

> Soon after another vehicle pulled up alongside, and someone in a passenger seat took out a gun and shot Herring in the head, Williams told police. I dont understand what Shotspotter has to do with this case at all. The question the case seems to hinge on is “who fired the gun”, not “was a gun fired at this location”. What am I missing?

dylan604 5y ago
>What am I missing? The fact that some of the evidence the police used to say who did what where was manufactured by the company.
- bryan0 5y ago
  The headline is: > A man spent a year in jail on a murder charge that hinged on disputed AI evidence. But after the reading the article, the evidence seemed irrelevant? Maybe the headline should have been: > AI evidence was manually reclassified after human review in murder case
  - DubiousPusher 5y ago
    Incorrect > Prosecutors said ShotSpotter picked up a gunshot sound where Williams was seen on surveillance camera footage in his car, putting it all forward as proof that Williams shot Herring right there and then. The police used the altered location of the shot to imply that the defendant's story about what happened wasn't true. Which was that another person in another var at a different location did the shooting. They used the altered location data alongside video evidence to imply the defendant and the victim were alone at the shot location and therefore the defendant was the only possible suspect.
    - bryan0 5y ago
      > The police used the altered location of the shot to imply that the defendant's story about what happened wasn't true. Which was that another person in another var at a different location did the shooting. Ok that would make more sense, but I definitely did not take that away from the article. I’m sure there are better sources on this case though.

arthur_sav 5y ago

Anyone else surprised that such a system is running across cities?

kevin_thibedeau 5y ago

I get that their system may sometimes need manual classification of signals but how is it possible to change the triangulated location? That is the core part of the technology and doesn't require ML to implement. Seems like it's all just Theranos style vapor.

c_weighted 5y ago
Locating on an echo at one more receivers rather than a perhaps quieter more direct path. Putting the wrong impulses together when computing a location. Computing the wrong start time for one or more impulses. If you want to read up on all the ways this kind of thing can go wrong, the technical term here is "source localization."
- kevin_thibedeau 5y ago
  The military has no problem detecting gunfire direction and estimating range from a single microphone array on a mast. Shotspotter is a distributed network of multiple sensors across a wide area. If they can't make it work after 15+ years then they are hopeless.
  - c_weighted 5y ago
    Boomerang is colocated with the target. Multipath is less likely to be an issue because there is likely to be line of sight to the source. Selecting the right impulses is easier because the spacing of the microphones provides some tight timing constraints. You’re not likely to see inconsistencies in timing the impulses lead to location error because any effects of the environment are likely to be consistent across the pulses. Finally, the location accuracy requirements are far different. ShotSpotter needs to put a dot on a map accurate enough to find shell casings or a victim in a wide geographic area. Boomerang just needs to give a good bearing from the receivers to return fire. The range doesn’t have to be all that precise, and the mathematics suggests it isn’t. These are two very different approaches to solve different problems.

backtoyoujim 5y ago

>“The problem? When we made it possible for anyone to generate art with artificial intelligence, barely anyone used it to make actual art. Instead, our AI model was forced to make videos for random inputs, trolling queries, and NSFW intents.” I am struggling with the use of the word "forced"; the researchers put their software out there to be used by anyone with an internet connection. The researchers created that dynamic. And how are they not seeing that AI created no-humans-used-to-make-this-porn could be a net good.

Vaslo 5y ago

We have this system in our city. Can’t speak to the accuracy and whether or not it should be admitted evidence. It has in some cases revealed the approximate locations of victims who have been shot where they potentially could have died without the AI heads up as to where they were.

TeeMassive 5y ago

Prosecution based on evidence provided by closed source and closed hardware based devices should never have been admitted in courts of laws in the first place.