30 ms·

Police are telling ShotSpotter to alter evidence from gunshot-detecting AI

516 points by danso 5y ago

dlgeek 5y ago
Ouch: > Over the years, ShotSpotter’s claims about its accuracy have increased, from 80 percent accurate to 90 percent accurate to 97 percent accurate. According to Greene, those numbers aren’t actually calculated by engineers, though. > “Our guarantee was put together by our sales and marketing department, not our engineers,” Greene told a San Francisco court in 2017. “We need to give them [customers] a number … We have to tell them something. … It’s not perfect. The dot on the map is simply a starting point.”
- ghostly_s 5y ago
  In fact, 89% of ShotSpotter deployments turn up no evidence of gun-related crime: [1] 1. https://endpolicesurveillance.com/ https://endpolicesurveillance.com/
  - bpodgursky 5y ago
    I'm not sure why this is an important stat. Isn't it good if there aren't gun crimes happening in a lot of places?
    - sodality2 5y ago
      Deployments I assume means police response, not deploying the system.
    - Cycl0ps 5y ago
      Check the link. What it's saying is that most times the police receive a notice of a gunshot, it's a false positive. In Chicago this leads to 60 dead-end responses each day. Very few systems are worth that trouble
      - handmodel 5y ago
        I 100% dont think it should be used as evidence but this seems like a decent rate for responding. There's 12,000 total officers in Chicago. Even if only 3,000 are on duty at any time that's not a huge inconvenience if there are still even dozens of cases a year you get to the scene 5 minutes before you would otherwise. If you track the number of calls to 911 that are "dead-ends" because there are no arrests I'm sure it would be high too. It also doesnt say the error rate for when the police aren't called. If criminals know that if they shoot a gun there is a 50% chance a police car will drive through 5 minutes later - wouldn't that deter you?
        Cycl0ps 5y ago
        Yeah, in fairness the steam controller had its perks. I didn't see the appeal of the two pads, but it was the first I can recall to do half/full pull triggers. Their customization also allowed you to do some crazy things. Personal favorite of mine was setting the gyroscope to act as right joystick, but only while zoomed in. This let you fine adjust your aim with about as much precision as with a mouse. In the end though every game had to be translated to the controller and there was always enough of a change to make it feel unintuitive. If Valve had paid an intern to go through the top games each day and make official layouts for games that roughly lined up with each other I'd probably be using one now.
  - adolph 5y ago
    > 89% of ShotSpotter deployments turn up no evidence of gun-related crime If ShotSpotter is limited to registering the the sound of gunshots, then it is limited to firearm-discharge-related crime which is a subset of gun-related crime.
  - meowface 5y ago
    That site says 14% of alerts result in a case report being filed and 10% result in a gun-related case report. But even a 10% alert -> case report rate is better than I would've expected for a big city. That still seems like it'd be extremely useful, even if it means some wasted man hours. And that's case reports - not necessarily detection true positives. If someone fires a gun at the sky and picks up the shell casings and drives away, with no witnesses or cameras and an inconclusive/indiscernible audio recording, police likely won't be able to file a report. Also, I'm guessing there are some confidence indicators for each alert: those case report rates probably don't capture the full story even when accounting for hypothetical inconclusive true positives. Police are likely (and should be) obligated to investigate every detection, including every detection rated as low-confidence. Plus that's just for Chicago. I doubt they actually achieve their claimed 97% accuracy anywhere, but it could be higher in other areas. Especially areas where loud noises are less common. The main issue seems to be when it's used as evidence. I think it shouldn't ever be admissible as evidence of anything. And to mitigate some of the other issues that site raises, it'd be nice if there were some kind of "no fishing expeditions" law, where police responding to a gunfire alert must exclusively look for evidence of a fired gun in that vicinity and not use it as an excuse to scavenge for low-hanging fruit by harassing / arresting people in the area for unrelated matters.
    - giantg2 5y ago
      "That still seems like it'd be extremely useful, even if it means some wasted man hours." I think the least wasteful approach would be to build rapport with the community and rely on their reports. Probably not an easy thing, but the most valuable things rarely are.
      - whimsicalism 5y ago
        Still doesn't stop the phenomena of everyone thinking "oh, someone else will call." Shotspotter is a very useful tool IMO.
        pessimizer 5y ago
        > the phenomena of everyone thinking "oh, someone else will call." This generally doesn't happen. It originated as a slander against the people who lived around where Kitty Genovese was killed, as an after-the-fact excuse for the lack of police response. > While there was no question that the attack occurred, and that some neighbors ignored cries for help, the portrayal of 38 witnesses as fully aware and unresponsive was erroneous. The article grossly exaggerated the number of witnesses and what they had perceived. None saw the attack in its entirety. Only a few had glimpsed parts of it, or recognized the cries for help. Many thought they had heard lovers or drunks quarreling. There were two attacks, not three. And afterward, two people did call the police. A 70-year-old woman ventured out and cradled the dying victim in her arms until they arrived. Ms. Genovese died on the way to a hospital. https://en.wikipedia.org/wiki/Murder_of_Kitty_Genovese#Accuracy_of_original_reports https://en.wikipedia.org/wiki/Murder_of_Kitty_Genovese#Accur...
  - GhostVII 5y ago
    Having an 11% success rate seems pretty impressive to me. Much better than the rate of false panic alarms, for example. And I'm sure a lot of those ShotSpotter alerts were accurate, but the police didn't get there soon enough.
    - handmodel 5y ago
      I think for the purposes of sending a car over to drive through a street its pretty good. The article discusses instances where it is used in trial and for that it is obviously pretty weak - surprised it can be used at all.
      - beerandt 5y ago
        I can see a use to confirm timing, or approx location, or as backing up a witnesses testimony. But it shouldn't be used as stand alone evidence.
    - giantg2 5y ago
      It shouldn't have anything to do with getting there soon enough. If they find shell casings, victims, or other evidence that a crime occurred (usually even discharging a firearm in a major city is considered a crime with very limited defenses), then they should still be collecting evidence and logging a report. Now, they may not be doing that because they don't want to have high crime numbers that are unsolved, or even dragging down the reputation of the safer areas (I've heard of officers doing this).
      - GhostVII 5y ago
        I was thinking of cases where someone fires a few shots and then everyone runs off - odds are the shell casings aren't going to be seen unless they do a careful sweep of the area, and the victims all left pretty quickly. No idea how often that actually happens though.
        giantg2 5y ago
        Yeah, but if the people running off aren't calling in, then the technology isn't going to make any impact until the public relations change.
- BoorishBears 5y ago
  That's the ouch? Not literally rewriting history to create evidence for a murder trial? > But after the 11:46 p.m. alert came in, a ShotSpotter analyst manually overrode the algorithms and “reclassified” the sound as a gunshot. Then, months later and after “post-processing,” another ShotSpotter analyst changed the alert’s coordinates to a location on South Stony Island Drive near where Williams’ car was seen on camera. > Motherboard’s review of court documents from the Williams case and other trials in Chicago and New York State, including testimony from ShotSpotter’s favored expert witness, suggests that the company’s analysts frequently modify alerts at the request of police departments—some of which appear to be grasping for evidence that supports their narrative of events. Marketing games are absolutely nothing compared to this.
  - klyrs 5y ago
    This sounds suspiciously like evidence tampering. If only the legal system would be so dogged in pursuit of justice for the cops & analysts involved in this conspiracy.
- pjungwir 5y ago
  Any time I see "accuracy" reported as a single number, I assume it's B.S. and "for marketing purposes only". If you're serious, give me the precision and recall.
  - the_snooze 5y ago
    >If you're serious, give me the precision and recall. The thing is decision-makers have no idea (and no care to know) what those are.
    - wcerfgba 5y ago
      Maybe these people shouldn't be decision-makers then. We need both greater transparency in decision-making processes, and greater education amongst the population about statistics, causal inference, and the scientific method. In order for the citizenry to hold decision-makers to account and obtain a higher standard we need: 1. Citizens to have the time and motivation to seek out and examine primary evidence. 2. Citizens to have the skills to understand and critique that evidence. 3. Robust mechanisms to allow citizens to replace people in power.
      - kube-system 5y ago
        This is an inherent problem with the specialization of labor. Those who run police departments have more relevant specializations to their job role than information science. This isn't a problem with statistics or science in particular -- it's an issue with every specialization. This is only going to get worse as popular opinion of the role of higher education is starting to shift from a more liberal-arts focus to a job-placement focus.
    - anigbrowl 5y ago
      Correct and that's unlikely to change, but it should be grounds to throw out the prosecution. Yes, this will make it harder to convict real criminals. Too bad.
  - drc500free 5y ago
    Precision is an attribute of the deployed system. The same tech in an affluent suburb will have different precision deployed in an inner city. Ran into this exact problem in biometrics, where NIST publishes tech error rates like FNMR and FMR, and system owners don't even know precision is a thing. They expect the vendor to tell them if it will work, when it depends how many true and false events you throw at the system.
- cryptonector 5y ago
  This is all devastating. These kinds of devices should use a a cryptographic tramper-evident log, and those logs should be available to the public in real time for archival purposes. That's the only way to avoid tampering. The same should be used for things like, e.g., rolls of who voted at every precinct.
- LegitShady 5y ago
  That sounds like fraud
avh02 5y ago
How hard would it be to create a community network of crappy microphones to do what shotspotter does? You have a million phones that all have gps and microphones
- Vaslo 5y ago
  Oh man the privacy folks are gonna hammer this comment hard.
  - Retric 5y ago
    I don’t think people firing guns have a reasonable expectation of privacy. Voluntarily submitting cellphone data is already used for several things like traffic congestion via Waze and related apps. I am vary pro privacy, but as long as it’s voluntary I don’t have an issue with this.
    - rspeele 5y ago
      It would depend strongly on whether the phone is analyzing the data from the mic and reporting to a server "possible gunshot heard, GPS coords X,Y, volume-level 60%, gunshot-like-score 75%" or whether the phone is sending raw recordings from the mic to a server and trusting that the server is only checking them for gunshot sound (not, say, saving your private conversations). And of course it's difficult for the average user of an app to tell the difference, or even for a tech savvy user to tell without investing some time and effort.
      - avh02 5y ago
        I'd not put that on my primary device either, maybe leave a spare plugged in near an open window, etc
      - jaegerpicker 5y ago
        Yes hopefully all the analysis would happen on device but that limits anything like a google home/alexa. Last I looked neither device allowed processing on device.
chrisseaton 5y ago
We used a shot detection system… in Afghanistan. Not really sure it’s needed in the United States.
Hokusai 5y ago
> Greene found a fifth shot, despite there being no physical evidence at the scene that Simmons had fired. Rochester police had also refused his multiple requests for them to test his hands and clothing for gunshot residue. This is the same that giving queues to a dog to bark to be able to inspect someone or a vehicle. New tech, old tricks. No technology will work if the people using it is corrupt.
- jaywalk 5y ago
  It's no wonder why the cops love it. Although the actual usefulness is questionable at best (much like the dogs) ShotSpotter will cook up "evidence" on command to help with the investigation (much like the dogs).
- slumdev 5y ago
  Any lawyers in the house who could comment on their refusal to test his hands? Can they refuse to run a test because they're afraid it might produce exculpatory evidence?
  - giantg2 5y ago
    Not a lawyer, but in my experience they don't need to do any tests. They usually have a policy that requires them to make a "thorough" investigation. Nobody holds them to that. For example, a trooper knew there were third party witnesses to a crime and did not seek them out. Granted, that trooper also withheld actual exculpatory evidence. Either way, nothing happened, because who watches the watchers?
  - tjfl 5y ago
    IANAL, but curious and searched. > A number of techniques designed to detect gunshot residues (GSR) on the hands of a suspect or victim have been developed. These techniques range from the now-discredited paraffin test to the more modern techniques which use instrumental analysis or scanning electron microscopy. The limitations of all GSR techniques are that the residues can be removed by rubbing or washing the hands and usually must be collected soon after the firearm is fired, yet even valid GSR tests are not conclusive. https://www.ojp.gov/ncjrs/virtual-library/abstracts/forensic-science-gunshot-residue-tests https://www.ojp.gov/ncjrs/virtual-library/abstracts/forensic...
mplewis 5y ago
Why is ShotSpotter using "AI" to analyze this? It's time-based triangulation of a loud noise. It must be nice to make a living on a grift like this.
- throwaway0a5e 5y ago
  Because gunshots are rare and if you're just triangulating loud noise you're gonna be overwhelmed by false positives. Even if you do basic filtration to try and tease out the sound of a gunshot vs everything else you're still gonna get overwhelmed by false positives.
  - deleted 5y ago
    [deleted]
  - dylan604 5y ago
    I don't know enough about the procedures, but would citizens calling 911 to report gun shots help coroborate and help weed out false positives?
    - tdeck 5y ago
      Many citizens can't tell the difference between gunshots and other explosive sounds like fireworks.
      - deleted 5y ago
        [deleted]
      - tolbish 5y ago
        If ShotSpotter can't either then it effectively useless.
        sneak 5y ago
        It's not useless, sometimes it can be used to fuel a prosecution against an innocent person who happens to be a racial minority. Police locking up black people in America is the reason police in America were created.
timy2shoes 5y ago
> But after the 11:46 p.m. alert came in, a ShotSpotter analyst manually overrode the algorithms and “reclassified” the sound as a gunshot. Then, months later and after “post-processing,” another ShotSpotter analyst changed the alert’s coordinates to a location on South Stony Island Drive near where Williams’ car was seen on camera. That's falsification of evidence. Will there be any consequences? Who watches the watchmen?
- jaywalk 5y ago
  You say falsification of evidence, they say manual review by an "expert" to correct errors from the automated process.
  - deleted 5y ago
    [deleted]
  - bluejekyll 5y ago
    Reclassifying the type of sound is possibly reasonable. Changing the coordinates seems like falsification of evidence. Later in the article it mentions adding a fifth shot in a different event. If shotspotter wants to be relied on for evidence, allowing technicians to go back and modify data seems like a recipe to quickly eliminate its use in cities and destroy its market share. Adding notes about manual review after the fact would be the only acceptable thing, but the originals should never be modified/deleted.
    - SV_BubbleTime 5y ago
      Sorry, but I’m an expert shooter. I can reasonably identify the round and sometimes the gun on the range next to me - I would say I have an excellently tuned ear for these things… And not a chance I could do that based on microphone playback between firework and gunshot with some random mic in a city. 1/2 the “trick” is knowing the location, at my range I know the echo of a 12ga vs a 45ACP, the crack difference in a 223 or a 6.5prc, if I went to some random place and you shot one random shot, it would be ridiculously less accurate. Shotspotter seems like a ton of bullshit to me after reading this article including… > Both the company and the Rochester Police Department “lost, deleted and/or destroyed the spool and/or other information containing sounds pertaining to the officer-involved shooting,”
Zigurd 5y ago
All technical support, including DNA, fingerprints, hair, fiber, etc. in law enforcement should be controlled outside of police departments and prosecutors, and should be made equally available to defendants as well as police and prosecutors. With the level of conflict of interest, you can't call it "science."
MisterTea 5y ago
Using privacy badger and noticing Vice pages now redirect to a 404 AFTER loading. Guess I'm not reading the article.
- bell-cot 5y ago
  Privacy Badger, Firefox, and NoScript here, and I can read the article fine. I'd suggest you try a few js-blocking add-ons.
steve76 5y ago
Correction: The lawful arrest of George Floyd. Jury was biased. Arsonists and a criminal mob influenced the judge. The lawful encounter with Adam Toledo. Criminals everywhere: change your ways or be destroyed.
hartator 5y ago
> Police said ShotSpotter, a surveillance system that uses hidden microphone sensors to detect the sound and location of gunshots. Wait, what?
oehpr 5y ago
This is tough. Pretty clearly what's happening here is shot spotter is producing a probability distribution about the places where a shot may have been fired, but they're not showing that information, they're showing a point. That's broken. It's a shame because setting up some microphones around the city to detect a firearm discharge seems like a great idea. The problem is people are overselling what it is doing. Something that jumps out at me in the article is this: Paul Greene, ShotSpotter’s expert witness and an employee of the company, testified at Simmons’ trial that “subsequently he was asked by the Rochester Police Department to essentially search and see if there were more shots fired than ShotSpotter picked up,” according to a civil lawsuit Simmons has filed against the city and the company. Greene found a fifth shot, despite there being no physical evidence at the scene that Simmons had fired. ... Like, what shotspotter is doing is attempting to resolve the origin of a sound wave in a city. The shockwave from a gun blast must reverberate across buildings all over the place, more than likely you're getting reflected waves rather than waves from the source. You're going to get echos and reverb and it's going to be messy and you're going to be imprecise. Would shotspotter's AI misclassify noises? Yah. Of course. If it's like any other classifier it's likely listing out a list of probabilities of what something is and, for the sake of simplicity, just saying it's the highest probability output. It's just a filtering step. If you know a gunshot occurred, you update your priors. Should shotspotter be able to detect the number of gunshots exactly? Maybe? How to differentiate between echos and original? I'm not shocked it can detect the wrong number of shots. Since it misclassifies noises, isn't exactly sure of the origin, and can read echos as additional shots, is it useless? No.
- slumdev 5y ago
  This is an unreliable system which police and prosecutors misrepresent as reliable. It might not be useless in the strictest sense of the word, but it ought to be taken away from them.
  - Frondo 5y ago
    This is the issue: it's unreliable, and its results presented as a matter of fact. This is also the latest I've heard of, in a long line of the criminal justice system presenting what's often closer to fiction as an indisputable fact of someone's guilt. One of the cases that made me incredibly angry was the junk arson science surrounding the conviction and execution of Cameron Todd Willingham in Texas in 2004: https://innocenceproject.org/cameron-todd-willingham-wrongfully-convicted-and-executed-in-texas/ https://innocenceproject.org/cameron-todd-willingham-wrongfu... But I also think of how unreliable eyewitness testimony is, and yet people's recollections of events (when we know, and have known for so long, that memory is plastic!), sometimes years later, are treated as fact: https://www.ncsc.org/trends/monthly-trends-articles/2017/the-trouble-with-eyewitness-identification-testimony-in-criminal-cases#:~:text=Research%20has%20found%20that%20eyewitness-identification%20testimony%20can%20be%20very%20unreliable.&text=Although%20witnesses%20can%20often%20be,most%20unreliable%20forms%20of%20evidence https://www.ncsc.org/trends/monthly-trends-articles/2017/the.... I don't know how to solve this, since the "proven" part of "innocent until proven guilty" seems like it's built on a house of cards, and the only people with a direct incentive to change it are suspects (i.e. the people that the criminal justice system has already decided could've done it, and how can you trust a criminal's motivations? eye roll)
bsanr 5y ago
>Motherboard recently obtained data demonstrating the stark racial disparity in how Chicago has deployed ShotSpotter. The sensors have been placed almost exclusively in predominantly Black and brown communities, while the white enclaves in the north and northwest of the city have no sensors at all, despite Chicago police data that shows gun crime is spread throughout the city. But I thought there was nothing racially-biased about American policing.
s1artibartfast 5y ago
On one hand, it seems that Shotspotter lacks the accuracy to be used in court. On the other hand, it seems like a valuable probabilistic tool to help police perform detective work and collect real admissible evidence. The problem seems to confusing the two. You can use an Ouija board to help find video evidence, but the video evidence still needs to show something incriminating.
- ChrisKnott 5y ago
  In the Williams case, I feel like the key information is how likely ShotSpotter is to miss a gunshot completely. They have video footage that shows the suspect and victim in the suspect's car alone at 11:46. If the victim was shot at 11:46 then this is extremely strong evidence of guilt, especially as it would also show the suspect lied. It seems reasonable to me that an algorithm might mistake a gunshot from inside a car as a firecracker out on the street. The classification aspect is not important. What the defence have to argue is not just that this sound is a firecracker, but the actual shot that killed the victim (which by the suspect's account happened in the same place), was not captured.
  - s1artibartfast 5y ago
    I'm not familiar enough with the case to guess at strategy. The keys seems to be: If police have any evidence besides ShotSpotter to connect the time and place of the shooting with the video of the two of them together.
    - ChrisKnott 5y ago
      I don't they necessarily need other evidence for the time. The article and most of the comments here are focussing on its inaccuracy in geolocation and classification, but it seems to be an accurate recording of loud sounds that ring out across the city. A high false positive rate actually works against the suspect. He needs it to have missed one.
      - s1artibartfast 5y ago
kyleblarson 5y ago
Vice conveniently leaves out the fact that Toledo had a gun in his hand with the slide locked back which means he had emptied the clip. There is video of this.
- ellenhp 5y ago
  Not to be pedantic but that's not what that means. That means that the slide was pulled back then released with the slide lock activated either manually or by an empty magazine. It's advisable to store pistols in this way with the magazine removed because it's generally the least threatening configuration a pistol can be in short of being disassembled.
- anigbrowl 5y ago
  Assumes facts not in evidence, to wit that he had a full clip at the outset of the incident.
zucked 5y ago
Shotspotter concerns me - the city I live in has paid millions for a system and recently expanded it. I am not aware of _any_ data that has been published from the system, even fuzzed data. That doesn't pass the sniff test for me.
- giantg2 5y ago
  Wow, millions?! Do they have any cost/benefit data for this system?
  - fnordfnordfnord 5y ago
    Of course not. They're hoping that a noise will some day coincide with a crime so then they can write a press release lauding the technology and the smart decision maker for buying it.
    - javajosh 5y ago
      The easiest way to be perceived as "doing something" is to spend money on it. Succeed or fail, you can still claim you tried. Much of the voting public is convinced that Democrats drink the blood of babies, so convincing them to look more closely and with skepticism is an uphill battle.
      - giantg2 5y ago
        "Succeed or fail, you can still claim you tried. Much of the voting public is convinced that Democrats drink the blood of babies, so convincing them to look more closely and with skepticism is an uphill battle." I'm not sure what you mean by this.
- anigbrowl 5y ago
  Find some like-minded people (perhaps including journalists), pool some resources, and use FOIA to sue. If your chief of police equivocates, call them a liar. Treat them like criminals, accuse them of running a protection racket.
zanderz 5y ago
Related: 11 years ago or so, 3 Tesla executives died when their small plane crashed in East Palo Alto, where the shotspotter system recorded audio of the crash: https://www.paloaltoonline.com/news/2010/02/19/shotspotter-system-records-tragic-plane-crash https://www.paloaltoonline.com/news/2010/02/19/shotspotter-s...
cratermoon 5y ago
Cops Lie. Here's a few examples. https://www.rollingstone.com/culture/culture-news/nypd-shake-shack-poisoned-cops-conspiracy-theory-1015897/ https://www.rollingstone.com/culture/culture-news/nypd-shake... https://popculture.com/trending/news/target-starbucks-tampon-no-suspicious-behavior-employees/ https://popculture.com/trending/news/target-starbucks-tampon... https://www.nytimes.com/2019/12/31/us/police-mcdonalds-coffee.html https://www.nytimes.com/2019/12/31/us/police-mcdonalds-coffe... https://www.washingtonpost.com/news/morning-mix/wp/2017/08/14/utah-subway-store-sues-police-after-worker-cleared-of-drugging-officer/ https://www.washingtonpost.com/news/morning-mix/wp/2017/08/1... https://popculture.com/trending/news/nypd-called-out-falsely-claiming-protesters-hiding-concrete-ice-cream-containers/ https://popculture.com/trending/news/nypd-called-out-falsely... https://www.oregonlive.com/crime/2019/07/police-lieutenant-wrote-he-firmly-believed-milkshakes-thrown-during-june-protest-contained-some-form-of-concrete.html https://www.oregonlive.com/crime/2019/07/police-lieutenant-w... https://www.foxla.com/news/deputies-arrest-kpcc-reporter-covering-demonstration-in-lynwood-after-compton-lasd-shooting https://www.foxla.com/news/deputies-arrest-kpcc-reporter-cov... https://www.courier-journal.com/story/news/local/breonna-taylor/2020/09/27/ballistics-report-dont-support-daniel-cameron-claim-breonna-taylor-boyfriend-shot-louisville-cop/3552007001/ https://www.courier-journal.com/story/news/local/breonna-tay... https://www.vice.com/en_us/article/939zda/how-stupid-do-cops-think-we-are-v27n1 https://www.vice.com/en_us/article/939zda/how-stupid-do-cops... https://www.vice.com/en_us/article/jmv94x/testilying-cops-are-liars-who-get-away-with-perjury https://www.vice.com/en_us/article/jmv94x/testilying-cops-ar... https://www.ocregister.com/2020/09/25/oc-sheriffs-deputies-who-lied-on-reports-testify-that-they-didnt-know-it-was-illegal/ https://www.ocregister.com/2020/09/25/oc-sheriffs-deputies-w... https://www.cjr.org/special_report/officials-say-chicago-police-joshua-beal.php https://www.cjr.org/special_report/officials-say-chicago-pol... https://fair.org/home/6-elements-of-police-spin-an-object-lesson-in-copspeak/ https://fair.org/home/6-elements-of-police-spin-an-object-le...
mabbo 5y ago
If we want to rely on this technology as being unbiased and accurate, then it must be a purely one-directional flow of information. Data should only flow from SpotShotter to the police, and never in the other direction. No questions, no inquiries, no anything. As soon as you do, you introduce bias or at least the ability for bias to exist, intentional or otherwise. If the SpotShotter technology is as good as they claim, there should never be a need. And if the police can't make a case with what SpotShotter gives them, they shouldn't rely on it.
- Decker87 5y ago
  It will always flow back, at least in the form of dollars spent.
  - notthemessiah 5y ago
    It also flows back in terms of political contributions, lobbying, and political appointments. Right now a former ShotSpotter VP is being nominated to lead the ATF.
- cryptonector 5y ago
  But also this should flow to the public in real-time for archival and should be tamper-evident to prevent alterations. Like a blockchain.
- fnordfnordfnord 5y ago
  > then it must be a purely one-directional flow of information. That's absolutely not possible.
garyfirestorm 5y ago
I don’t understand how they get to just walk away without any consequences. prosecutor1: sir we got evidence that defendant1 was behind the shooting defendant1: sir I was miles away from this area - let’s verify the evidence prosecutor1: you know what, never mind - poker face Court: poker face Defendant1: poker face >“Rather than defend the evidence, [prosecutors] just ran away from it,” he said.
- LatteLazy 5y ago
  Worse: this is one of the most discriminatory issues in the criminal justice system. The prosecutor can and will use that "evidence" unless and until the defence challenges it. But poor people with overworked lawyers don't have the time to challenge anything so they go down based on BS. The same thing happens with Stingrays apparently: as soon as someone asks where the lead came from for the arrest, its case dropped. But if you can't afford a lawyer with time to chase down every detail you go to jail on illegal evidence.
pmarreck 5y ago
There needs to be a law that any technology used by law enforcement in the course of doing business is open-source. I don't see how this isn't already the case given the lives at stake
rhizome 5y ago
For some reason this isn't a felony offense by anybody involved.
javajosh 5y ago
Why didn't the judge find the prosecution in contempt of court? And why didn't this trigger an internal investigation inside the Chicago PD? Modifying evidence to fit a crime is itself a crime, withdrawn or not, and heads should be rolling right now.
- sneak 5y ago
  The courts and the prosecutors and the police are all on the same team. The adversarial, truth-seeking system you believe that this is is a fiction. It's a pipeline for money, nothing more.
vmception 5y ago
> The sensors have been placed almost exclusively in predominantly Black and brown communities, while the white enclaves in the north and northwest of the city have no sensors at all, despite Chicago police data that shows gun crime is spread throughout the city. One country, two systems Arbitrary death by firing squad? Check Arbitrary expropriation of private property? Check Arbitrary internment for one class of society? Recurring theme, but Check
noobermin 5y ago
Stories like this are part and parcel why many people don't trust the police.
toofy 5y ago
> ShotSpotter’s claims about its accuracy have increased, from 80 percent accurate to 90 percent accurate to 97 percent accurate. According to Greene, those numbers aren’t actually calculated by engineers, though. > “Our guarantee was put together by our sales and marketing department, not our engineers,” Greene told a San Francisco court in 2017. “We need to give them [customers] a number … We have to tell them something. … there’s the problem right there. a large number of us have seen, first hand, how a sales and marketing team will happily distort the abilities of something we build. and we’ve all seen first hand how clients will lap that up and then turn around and blame the engineers when it doesn’t live up to what they were led to believe. as his quote says, those numbers aren’t actually calculated by engineers, they’re calculated by the sales and marketing department and he happily pushes this nonsense because “… we have to tell them something.” meanwhile people are dying from this/being prosecuted and the accuracy rates are pulled out of thin air by the marketing/sales department.
- at-fates-hands 5y ago
  >> and then turn around and blame the engineers when it doesn’t live up to what they were led to believe. Any examples of how shotspotter technology has not lived up to or has failed in what it was designed to do? I'm not really that familiar with the technology, all I know is that if there are shots fired within the vicinity of a shotspotter it registers those shots. Does it have the capability of triangulating exactly where the shot took place? We have them all over Minneapolis and there's several twitter accounts that monitor police activity and I've seen a lot of posts saying: shotspotter was activated at 33rd and Hennepin, police located several bullet casings nearby and are actively searching for the shooter Stuff like that. What's the difference between having 80% and 97% accuracy?
  - ashtonkem 5y ago
    > Any examples of how shotspotter technology has not lived up to or has failed in what it was designed to do? You're literally quoting on an article where they moved the location of a sound by over a mile, and reclassified it from a firework to a gunshot. They tried to use this obviously falsified evidence to throw a man in jail, and had to withdraw the evidence once this was pointed out. Sure seems like an example of it not living up to what it was designed to do. Even if it turns out Williams is guilty and he did shoot that man, Shotspotter was just blatantly wrong about what happened and where it happened. Also, 86% of the time it detects a gunshot police find nothing. https://chicago.suntimes.com/2021/5/4/22417660/shotspotter-analysis-macarthur-justice-center-chicago-police-chicago-gun-violence-editorial https://chicago.suntimes.com/2021/5/4/22417660/shotspotter-a... > We have them all over Minneapolis and there's several twitter accounts that monitor police activity and I've seen a lot of posts saying: Ooookay? This is practically the definition of how the plurality of anecdote is not data. > What's the difference between having 80% and 97% accuracy? When it's being used to send armed representatives of the state to arrest and prosecute people? The difference is a lot of people arrested and prosecuted based on faulty evidence.
advael 5y ago
All you need to hear from this article is that no one outside of the company can independently audit the software Any such solution should be dead in the water as a source of meaningful forensic evidence, period
mikewarot 5y ago
Why is this considered "evidence" at all? Raw recordings of sound, with timestamps and locations of sensors, could be evidence, but something output from an AI? Let's also consider that the speed of sound isn't constant, sound reflects off of everything, and can even be ducted by natural causes.
sennight 5y ago
> Rochester police had also refused his multiple requests for them to test his hands and clothing for gunshot residue. GSR tests are total crap and in my experience lean very hard toward the false positive side of things, unsurprisingly nitrates are all over the place. Cops know this, so them not administering a test that can only hurt a suspect's chances with a jury - well the guys on the street certainly weren't out to frame him.
tim333 5y ago
There must be some laws against doing this stuff in the US? In the uk we have 'perverting the course of justice'.

516 points by danso 5y ago

Ouch: > Over the years, ShotSpotter’s claims about its accuracy have increased, from 80 percent accurate to 90 percent accurate to 97 percent accurate. According to Greene, those numbers aren’t actually calculated by engineers, though. > “Our guarantee was put together by our sales and marketing department, not our engineers,” Greene told a San Francisco court in 2017. “We need to give them [customers] a number … We have to tell them something. … It’s not perfect. The dot on the map is simply a starting point.”

ghostly_s 5y ago
In fact, 89% of ShotSpotter deployments turn up no evidence of gun-related crime: [1] 1. https://endpolicesurveillance.com/ https://endpolicesurveillance.com/
- bpodgursky 5y ago
  I'm not sure why this is an important stat. Isn't it good if there aren't gun crimes happening in a lot of places?
  - sodality2 5y ago
    Deployments I assume means police response, not deploying the system.
  - Cycl0ps 5y ago
    Check the link. What it's saying is that most times the police receive a notice of a gunshot, it's a false positive. In Chicago this leads to 60 dead-end responses each day. Very few systems are worth that trouble
    - handmodel 5y ago
      I 100% dont think it should be used as evidence but this seems like a decent rate for responding. There's 12,000 total officers in Chicago. Even if only 3,000 are on duty at any time that's not a huge inconvenience if there are still even dozens of cases a year you get to the scene 5 minutes before you would otherwise. If you track the number of calls to 911 that are "dead-ends" because there are no arrests I'm sure it would be high too. It also doesnt say the error rate for when the police aren't called. If criminals know that if they shoot a gun there is a 50% chance a police car will drive through 5 minutes later - wouldn't that deter you?
      - Cycl0ps 5y ago
        Yeah, in fairness the steam controller had its perks. I didn't see the appeal of the two pads, but it was the first I can recall to do half/full pull triggers. Their customization also allowed you to do some crazy things. Personal favorite of mine was setting the gyroscope to act as right joystick, but only while zoomed in. This let you fine adjust your aim with about as much precision as with a mouse. In the end though every game had to be translated to the controller and there was always enough of a change to make it feel unintuitive. If Valve had paid an intern to go through the top games each day and make official layouts for games that roughly lined up with each other I'd probably be using one now.
- adolph 5y ago
  > 89% of ShotSpotter deployments turn up no evidence of gun-related crime If ShotSpotter is limited to registering the the sound of gunshots, then it is limited to firearm-discharge-related crime which is a subset of gun-related crime.
- meowface 5y ago
  That site says 14% of alerts result in a case report being filed and 10% result in a gun-related case report. But even a 10% alert -> case report rate is better than I would've expected for a big city. That still seems like it'd be extremely useful, even if it means some wasted man hours. And that's case reports - not necessarily detection true positives. If someone fires a gun at the sky and picks up the shell casings and drives away, with no witnesses or cameras and an inconclusive/indiscernible audio recording, police likely won't be able to file a report. Also, I'm guessing there are some confidence indicators for each alert: those case report rates probably don't capture the full story even when accounting for hypothetical inconclusive true positives. Police are likely (and should be) obligated to investigate every detection, including every detection rated as low-confidence. Plus that's just for Chicago. I doubt they actually achieve their claimed 97% accuracy anywhere, but it could be higher in other areas. Especially areas where loud noises are less common. The main issue seems to be when it's used as evidence. I think it shouldn't ever be admissible as evidence of anything. And to mitigate some of the other issues that site raises, it'd be nice if there were some kind of "no fishing expeditions" law, where police responding to a gunfire alert must exclusively look for evidence of a fired gun in that vicinity and not use it as an excuse to scavenge for low-hanging fruit by harassing / arresting people in the area for unrelated matters.
  - giantg2 5y ago
    "That still seems like it'd be extremely useful, even if it means some wasted man hours." I think the least wasteful approach would be to build rapport with the community and rely on their reports. Probably not an easy thing, but the most valuable things rarely are.
    - whimsicalism 5y ago
      Still doesn't stop the phenomena of everyone thinking "oh, someone else will call." Shotspotter is a very useful tool IMO.
      - pessimizer 5y ago
        > the phenomena of everyone thinking "oh, someone else will call." This generally doesn't happen. It originated as a slander against the people who lived around where Kitty Genovese was killed, as an after-the-fact excuse for the lack of police response. > While there was no question that the attack occurred, and that some neighbors ignored cries for help, the portrayal of 38 witnesses as fully aware and unresponsive was erroneous. The article grossly exaggerated the number of witnesses and what they had perceived. None saw the attack in its entirety. Only a few had glimpsed parts of it, or recognized the cries for help. Many thought they had heard lovers or drunks quarreling. There were two attacks, not three. And afterward, two people did call the police. A 70-year-old woman ventured out and cradled the dying victim in her arms until they arrived. Ms. Genovese died on the way to a hospital. https://en.wikipedia.org/wiki/Murder_of_Kitty_Genovese#Accuracy_of_original_reports https://en.wikipedia.org/wiki/Murder_of_Kitty_Genovese#Accur...
- GhostVII 5y ago
  Having an 11% success rate seems pretty impressive to me. Much better than the rate of false panic alarms, for example. And I'm sure a lot of those ShotSpotter alerts were accurate, but the police didn't get there soon enough.
  - handmodel 5y ago
    I think for the purposes of sending a car over to drive through a street its pretty good. The article discusses instances where it is used in trial and for that it is obviously pretty weak - surprised it can be used at all.
    - beerandt 5y ago
      I can see a use to confirm timing, or approx location, or as backing up a witnesses testimony. But it shouldn't be used as stand alone evidence.
  - giantg2 5y ago
    It shouldn't have anything to do with getting there soon enough. If they find shell casings, victims, or other evidence that a crime occurred (usually even discharging a firearm in a major city is considered a crime with very limited defenses), then they should still be collecting evidence and logging a report. Now, they may not be doing that because they don't want to have high crime numbers that are unsolved, or even dragging down the reputation of the safer areas (I've heard of officers doing this).
    - GhostVII 5y ago
      I was thinking of cases where someone fires a few shots and then everyone runs off - odds are the shell casings aren't going to be seen unless they do a careful sweep of the area, and the victims all left pretty quickly. No idea how often that actually happens though.
      - giantg2 5y ago
        Yeah, but if the people running off aren't calling in, then the technology isn't going to make any impact until the public relations change.
BoorishBears 5y ago
That's the ouch? Not literally rewriting history to create evidence for a murder trial? > But after the 11:46 p.m. alert came in, a ShotSpotter analyst manually overrode the algorithms and “reclassified” the sound as a gunshot. Then, months later and after “post-processing,” another ShotSpotter analyst changed the alert’s coordinates to a location on South Stony Island Drive near where Williams’ car was seen on camera. > Motherboard’s review of court documents from the Williams case and other trials in Chicago and New York State, including testimony from ShotSpotter’s favored expert witness, suggests that the company’s analysts frequently modify alerts at the request of police departments—some of which appear to be grasping for evidence that supports their narrative of events. Marketing games are absolutely nothing compared to this.
- klyrs 5y ago
  This sounds suspiciously like evidence tampering. If only the legal system would be so dogged in pursuit of justice for the cops & analysts involved in this conspiracy.
pjungwir 5y ago
Any time I see "accuracy" reported as a single number, I assume it's B.S. and "for marketing purposes only". If you're serious, give me the precision and recall.
- the_snooze 5y ago
  >If you're serious, give me the precision and recall. The thing is decision-makers have no idea (and no care to know) what those are.
  - wcerfgba 5y ago
    Maybe these people shouldn't be decision-makers then. We need both greater transparency in decision-making processes, and greater education amongst the population about statistics, causal inference, and the scientific method. In order for the citizenry to hold decision-makers to account and obtain a higher standard we need: 1. Citizens to have the time and motivation to seek out and examine primary evidence. 2. Citizens to have the skills to understand and critique that evidence. 3. Robust mechanisms to allow citizens to replace people in power.
    - kube-system 5y ago
      This is an inherent problem with the specialization of labor. Those who run police departments have more relevant specializations to their job role than information science. This isn't a problem with statistics or science in particular -- it's an issue with every specialization. This is only going to get worse as popular opinion of the role of higher education is starting to shift from a more liberal-arts focus to a job-placement focus.
  - anigbrowl 5y ago
    Correct and that's unlikely to change, but it should be grounds to throw out the prosecution. Yes, this will make it harder to convict real criminals. Too bad.
- drc500free 5y ago
  Precision is an attribute of the deployed system. The same tech in an affluent suburb will have different precision deployed in an inner city. Ran into this exact problem in biometrics, where NIST publishes tech error rates like FNMR and FMR, and system owners don't even know precision is a thing. They expect the vendor to tell them if it will work, when it depends how many true and false events you throw at the system.
cryptonector 5y ago
This is all devastating. These kinds of devices should use a a cryptographic tramper-evident log, and those logs should be available to the public in real time for archival purposes. That's the only way to avoid tampering. The same should be used for things like, e.g., rolls of who voted at every precinct.
LegitShady 5y ago
That sounds like fraud

avh02 5y ago

How hard would it be to create a community network of crappy microphones to do what shotspotter does? You have a million phones that all have gps and microphones

Vaslo 5y ago
Oh man the privacy folks are gonna hammer this comment hard.
- Retric 5y ago
  I don’t think people firing guns have a reasonable expectation of privacy. Voluntarily submitting cellphone data is already used for several things like traffic congestion via Waze and related apps. I am vary pro privacy, but as long as it’s voluntary I don’t have an issue with this.
  - rspeele 5y ago
    It would depend strongly on whether the phone is analyzing the data from the mic and reporting to a server "possible gunshot heard, GPS coords X,Y, volume-level 60%, gunshot-like-score 75%" or whether the phone is sending raw recordings from the mic to a server and trusting that the server is only checking them for gunshot sound (not, say, saving your private conversations). And of course it's difficult for the average user of an app to tell the difference, or even for a tech savvy user to tell without investing some time and effort.
    - avh02 5y ago
      I'd not put that on my primary device either, maybe leave a spare plugged in near an open window, etc
    - jaegerpicker 5y ago
      Yes hopefully all the analysis would happen on device but that limits anything like a google home/alexa. Last I looked neither device allowed processing on device.

chrisseaton 5y ago

We used a shot detection system… in Afghanistan. Not really sure it’s needed in the United States.

Hokusai 5y ago

> Greene found a fifth shot, despite there being no physical evidence at the scene that Simmons had fired. Rochester police had also refused his multiple requests for them to test his hands and clothing for gunshot residue. This is the same that giving queues to a dog to bark to be able to inspect someone or a vehicle. New tech, old tricks. No technology will work if the people using it is corrupt.

jaywalk 5y ago
It's no wonder why the cops love it. Although the actual usefulness is questionable at best (much like the dogs) ShotSpotter will cook up "evidence" on command to help with the investigation (much like the dogs).
slumdev 5y ago
Any lawyers in the house who could comment on their refusal to test his hands? Can they refuse to run a test because they're afraid it might produce exculpatory evidence?
- giantg2 5y ago
  Not a lawyer, but in my experience they don't need to do any tests. They usually have a policy that requires them to make a "thorough" investigation. Nobody holds them to that. For example, a trooper knew there were third party witnesses to a crime and did not seek them out. Granted, that trooper also withheld actual exculpatory evidence. Either way, nothing happened, because who watches the watchers?
- tjfl 5y ago
  IANAL, but curious and searched. > A number of techniques designed to detect gunshot residues (GSR) on the hands of a suspect or victim have been developed. These techniques range from the now-discredited paraffin test to the more modern techniques which use instrumental analysis or scanning electron microscopy. The limitations of all GSR techniques are that the residues can be removed by rubbing or washing the hands and usually must be collected soon after the firearm is fired, yet even valid GSR tests are not conclusive. https://www.ojp.gov/ncjrs/virtual-library/abstracts/forensic-science-gunshot-residue-tests https://www.ojp.gov/ncjrs/virtual-library/abstracts/forensic...

mplewis 5y ago

Why is ShotSpotter using "AI" to analyze this? It's time-based triangulation of a loud noise. It must be nice to make a living on a grift like this.

throwaway0a5e 5y ago
Because gunshots are rare and if you're just triangulating loud noise you're gonna be overwhelmed by false positives. Even if you do basic filtration to try and tease out the sound of a gunshot vs everything else you're still gonna get overwhelmed by false positives.
- deleted 5y ago
  [deleted]
- dylan604 5y ago
  I don't know enough about the procedures, but would citizens calling 911 to report gun shots help coroborate and help weed out false positives?
  - tdeck 5y ago
    Many citizens can't tell the difference between gunshots and other explosive sounds like fireworks.
    - deleted 5y ago
      [deleted]
    - tolbish 5y ago
      If ShotSpotter can't either then it effectively useless.
      - sneak 5y ago
        It's not useless, sometimes it can be used to fuel a prosecution against an innocent person who happens to be a racial minority. Police locking up black people in America is the reason police in America were created.

timy2shoes 5y ago

> But after the 11:46 p.m. alert came in, a ShotSpotter analyst manually overrode the algorithms and “reclassified” the sound as a gunshot. Then, months later and after “post-processing,” another ShotSpotter analyst changed the alert’s coordinates to a location on South Stony Island Drive near where Williams’ car was seen on camera. That's falsification of evidence. Will there be any consequences? Who watches the watchmen?

jaywalk 5y ago
You say falsification of evidence, they say manual review by an "expert" to correct errors from the automated process.
- deleted 5y ago
  [deleted]
- bluejekyll 5y ago
  Reclassifying the type of sound is possibly reasonable. Changing the coordinates seems like falsification of evidence. Later in the article it mentions adding a fifth shot in a different event. If shotspotter wants to be relied on for evidence, allowing technicians to go back and modify data seems like a recipe to quickly eliminate its use in cities and destroy its market share. Adding notes about manual review after the fact would be the only acceptable thing, but the originals should never be modified/deleted.
  - SV_BubbleTime 5y ago
    Sorry, but I’m an expert shooter. I can reasonably identify the round and sometimes the gun on the range next to me - I would say I have an excellently tuned ear for these things… And not a chance I could do that based on microphone playback between firework and gunshot with some random mic in a city. 1/2 the “trick” is knowing the location, at my range I know the echo of a 12ga vs a 45ACP, the crack difference in a 223 or a 6.5prc, if I went to some random place and you shot one random shot, it would be ridiculously less accurate. Shotspotter seems like a ton of bullshit to me after reading this article including… > Both the company and the Rochester Police Department “lost, deleted and/or destroyed the spool and/or other information containing sounds pertaining to the officer-involved shooting,”

Zigurd 5y ago

All technical support, including DNA, fingerprints, hair, fiber, etc. in law enforcement should be controlled outside of police departments and prosecutors, and should be made equally available to defendants as well as police and prosecutors. With the level of conflict of interest, you can't call it "science."

MisterTea 5y ago

Using privacy badger and noticing Vice pages now redirect to a 404 AFTER loading. Guess I'm not reading the article.

bell-cot 5y ago
Privacy Badger, Firefox, and NoScript here, and I can read the article fine. I'd suggest you try a few js-blocking add-ons.

steve76 5y ago

Correction: The lawful arrest of George Floyd. Jury was biased. Arsonists and a criminal mob influenced the judge. The lawful encounter with Adam Toledo. Criminals everywhere: change your ways or be destroyed.

hartator 5y ago

> Police said ShotSpotter, a surveillance system that uses hidden microphone sensors to detect the sound and location of gunshots. Wait, what?

oehpr 5y ago

This is tough. Pretty clearly what's happening here is shot spotter is producing a probability distribution about the places where a shot may have been fired, but they're not showing that information, they're showing a point. That's broken. It's a shame because setting up some microphones around the city to detect a firearm discharge seems like a great idea. The problem is people are overselling what it is doing. Something that jumps out at me in the article is this: Paul Greene, ShotSpotter’s expert witness and an employee of the company, testified at Simmons’ trial that “subsequently he was asked by the Rochester Police Department to essentially search and see if there were more shots fired than ShotSpotter picked up,” according to a civil lawsuit Simmons has filed against the city and the company. Greene found a fifth shot, despite there being no physical evidence at the scene that Simmons had fired. ... Like, what shotspotter is doing is attempting to resolve the origin of a sound wave in a city. The shockwave from a gun blast must reverberate across buildings all over the place, more than likely you're getting reflected waves rather than waves from the source. You're going to get echos and reverb and it's going to be messy and you're going to be imprecise. Would shotspotter's AI misclassify noises? Yah. Of course. If it's like any other classifier it's likely listing out a list of probabilities of what something is and, for the sake of simplicity, just saying it's the highest probability output. It's just a filtering step. If you know a gunshot occurred, you update your priors. Should shotspotter be able to detect the number of gunshots exactly? Maybe? How to differentiate between echos and original? I'm not shocked it can detect the wrong number of shots. Since it misclassifies noises, isn't exactly sure of the origin, and can read echos as additional shots, is it useless? No.

slumdev 5y ago
This is an unreliable system which police and prosecutors misrepresent as reliable. It might not be useless in the strictest sense of the word, but it ought to be taken away from them.
- Frondo 5y ago
  This is the issue: it's unreliable, and its results presented as a matter of fact. This is also the latest I've heard of, in a long line of the criminal justice system presenting what's often closer to fiction as an indisputable fact of someone's guilt. One of the cases that made me incredibly angry was the junk arson science surrounding the conviction and execution of Cameron Todd Willingham in Texas in 2004: https://innocenceproject.org/cameron-todd-willingham-wrongfully-convicted-and-executed-in-texas/ https://innocenceproject.org/cameron-todd-willingham-wrongfu... But I also think of how unreliable eyewitness testimony is, and yet people's recollections of events (when we know, and have known for so long, that memory is plastic!), sometimes years later, are treated as fact: https://www.ncsc.org/trends/monthly-trends-articles/2017/the-trouble-with-eyewitness-identification-testimony-in-criminal-cases#:~:text=Research%20has%20found%20that%20eyewitness-identification%20testimony%20can%20be%20very%20unreliable.&text=Although%20witnesses%20can%20often%20be,most%20unreliable%20forms%20of%20evidence https://www.ncsc.org/trends/monthly-trends-articles/2017/the.... I don't know how to solve this, since the "proven" part of "innocent until proven guilty" seems like it's built on a house of cards, and the only people with a direct incentive to change it are suspects (i.e. the people that the criminal justice system has already decided could've done it, and how can you trust a criminal's motivations? eye roll)

bsanr 5y ago

>Motherboard recently obtained data demonstrating the stark racial disparity in how Chicago has deployed ShotSpotter. The sensors have been placed almost exclusively in predominantly Black and brown communities, while the white enclaves in the north and northwest of the city have no sensors at all, despite Chicago police data that shows gun crime is spread throughout the city. But I thought there was nothing racially-biased about American policing.

s1artibartfast 5y ago

On one hand, it seems that Shotspotter lacks the accuracy to be used in court. On the other hand, it seems like a valuable probabilistic tool to help police perform detective work and collect real admissible evidence. The problem seems to confusing the two. You can use an Ouija board to help find video evidence, but the video evidence still needs to show something incriminating.

ChrisKnott 5y ago
In the Williams case, I feel like the key information is how likely ShotSpotter is to miss a gunshot completely. They have video footage that shows the suspect and victim in the suspect's car alone at 11:46. If the victim was shot at 11:46 then this is extremely strong evidence of guilt, especially as it would also show the suspect lied. It seems reasonable to me that an algorithm might mistake a gunshot from inside a car as a firecracker out on the street. The classification aspect is not important. What the defence have to argue is not just that this sound is a firecracker, but the actual shot that killed the victim (which by the suspect's account happened in the same place), was not captured.
- s1artibartfast 5y ago
  I'm not familiar enough with the case to guess at strategy. The keys seems to be: If police have any evidence besides ShotSpotter to connect the time and place of the shooting with the video of the two of them together.
  - ChrisKnott 5y ago
    I don't they necessarily need other evidence for the time. The article and most of the comments here are focussing on its inaccuracy in geolocation and classification, but it seems to be an accurate recording of loud sounds that ring out across the city. A high false positive rate actually works against the suspect. He needs it to have missed one.
    - s1artibartfast 5y ago

kyleblarson 5y ago

Vice conveniently leaves out the fact that Toledo had a gun in his hand with the slide locked back which means he had emptied the clip. There is video of this.

ellenhp 5y ago
Not to be pedantic but that's not what that means. That means that the slide was pulled back then released with the slide lock activated either manually or by an empty magazine. It's advisable to store pistols in this way with the magazine removed because it's generally the least threatening configuration a pistol can be in short of being disassembled.
anigbrowl 5y ago
Assumes facts not in evidence, to wit that he had a full clip at the outset of the incident.

zucked 5y ago

Shotspotter concerns me - the city I live in has paid millions for a system and recently expanded it. I am not aware of _any_ data that has been published from the system, even fuzzed data. That doesn't pass the sniff test for me.

giantg2 5y ago
Wow, millions?! Do they have any cost/benefit data for this system?
- fnordfnordfnord 5y ago
  Of course not. They're hoping that a noise will some day coincide with a crime so then they can write a press release lauding the technology and the smart decision maker for buying it.
  - javajosh 5y ago
    The easiest way to be perceived as "doing something" is to spend money on it. Succeed or fail, you can still claim you tried. Much of the voting public is convinced that Democrats drink the blood of babies, so convincing them to look more closely and with skepticism is an uphill battle.
    - giantg2 5y ago
      "Succeed or fail, you can still claim you tried. Much of the voting public is convinced that Democrats drink the blood of babies, so convincing them to look more closely and with skepticism is an uphill battle." I'm not sure what you mean by this.
anigbrowl 5y ago
Find some like-minded people (perhaps including journalists), pool some resources, and use FOIA to sue. If your chief of police equivocates, call them a liar. Treat them like criminals, accuse them of running a protection racket.

zanderz 5y ago

Related: 11 years ago or so, 3 Tesla executives died when their small plane crashed in East Palo Alto, where the shotspotter system recorded audio of the crash: https://www.paloaltoonline.com/news/2010/02/19/shotspotter-system-records-tragic-plane-crash https://www.paloaltoonline.com/news/2010/02/19/shotspotter-s...

cratermoon 5y ago

Cops Lie. Here's a few examples. https://www.rollingstone.com/culture/culture-news/nypd-shake-shack-poisoned-cops-conspiracy-theory-1015897/ https://www.rollingstone.com/culture/culture-news/nypd-shake... https://popculture.com/trending/news/target-starbucks-tampon-no-suspicious-behavior-employees/ https://popculture.com/trending/news/target-starbucks-tampon... https://www.nytimes.com/2019/12/31/us/police-mcdonalds-coffee.html https://www.nytimes.com/2019/12/31/us/police-mcdonalds-coffe... https://www.washingtonpost.com/news/morning-mix/wp/2017/08/14/utah-subway-store-sues-police-after-worker-cleared-of-drugging-officer/ https://www.washingtonpost.com/news/morning-mix/wp/2017/08/1... https://popculture.com/trending/news/nypd-called-out-falsely-claiming-protesters-hiding-concrete-ice-cream-containers/ https://popculture.com/trending/news/nypd-called-out-falsely... https://www.oregonlive.com/crime/2019/07/police-lieutenant-wrote-he-firmly-believed-milkshakes-thrown-during-june-protest-contained-some-form-of-concrete.html https://www.oregonlive.com/crime/2019/07/police-lieutenant-w... https://www.foxla.com/news/deputies-arrest-kpcc-reporter-covering-demonstration-in-lynwood-after-compton-lasd-shooting https://www.foxla.com/news/deputies-arrest-kpcc-reporter-cov... https://www.courier-journal.com/story/news/local/breonna-taylor/2020/09/27/ballistics-report-dont-support-daniel-cameron-claim-breonna-taylor-boyfriend-shot-louisville-cop/3552007001/ https://www.courier-journal.com/story/news/local/breonna-tay... https://www.vice.com/en_us/article/939zda/how-stupid-do-cops-think-we-are-v27n1 https://www.vice.com/en_us/article/939zda/how-stupid-do-cops... https://www.vice.com/en_us/article/jmv94x/testilying-cops-are-liars-who-get-away-with-perjury https://www.vice.com/en_us/article/jmv94x/testilying-cops-ar... https://www.ocregister.com/2020/09/25/oc-sheriffs-deputies-who-lied-on-reports-testify-that-they-didnt-know-it-was-illegal/ https://www.ocregister.com/2020/09/25/oc-sheriffs-deputies-w... https://www.cjr.org/special_report/officials-say-chicago-police-joshua-beal.php https://www.cjr.org/special_report/officials-say-chicago-pol... https://fair.org/home/6-elements-of-police-spin-an-object-lesson-in-copspeak/ https://fair.org/home/6-elements-of-police-spin-an-object-le...

mabbo 5y ago

If we want to rely on this technology as being unbiased and accurate, then it must be a purely one-directional flow of information. Data should only flow from SpotShotter to the police, and never in the other direction. No questions, no inquiries, no anything. As soon as you do, you introduce bias or at least the ability for bias to exist, intentional or otherwise. If the SpotShotter technology is as good as they claim, there should never be a need. And if the police can't make a case with what SpotShotter gives them, they shouldn't rely on it.

Decker87 5y ago
It will always flow back, at least in the form of dollars spent.
- notthemessiah 5y ago
  It also flows back in terms of political contributions, lobbying, and political appointments. Right now a former ShotSpotter VP is being nominated to lead the ATF.
cryptonector 5y ago
But also this should flow to the public in real-time for archival and should be tamper-evident to prevent alterations. Like a blockchain.
fnordfnordfnord 5y ago
> then it must be a purely one-directional flow of information. That's absolutely not possible.

garyfirestorm 5y ago

I don’t understand how they get to just walk away without any consequences. prosecutor1: sir we got evidence that defendant1 was behind the shooting defendant1: sir I was miles away from this area - let’s verify the evidence prosecutor1: you know what, never mind - poker face Court: poker face Defendant1: poker face >“Rather than defend the evidence, [prosecutors] just ran away from it,” he said.

LatteLazy 5y ago
Worse: this is one of the most discriminatory issues in the criminal justice system. The prosecutor can and will use that "evidence" unless and until the defence challenges it. But poor people with overworked lawyers don't have the time to challenge anything so they go down based on BS. The same thing happens with Stingrays apparently: as soon as someone asks where the lead came from for the arrest, its case dropped. But if you can't afford a lawyer with time to chase down every detail you go to jail on illegal evidence.

pmarreck 5y ago

There needs to be a law that any technology used by law enforcement in the course of doing business is open-source. I don't see how this isn't already the case given the lives at stake

rhizome 5y ago

For some reason this isn't a felony offense by anybody involved.

javajosh 5y ago

Why didn't the judge find the prosecution in contempt of court? And why didn't this trigger an internal investigation inside the Chicago PD? Modifying evidence to fit a crime is itself a crime, withdrawn or not, and heads should be rolling right now.

sneak 5y ago
The courts and the prosecutors and the police are all on the same team. The adversarial, truth-seeking system you believe that this is is a fiction. It's a pipeline for money, nothing more.

vmception 5y ago

> The sensors have been placed almost exclusively in predominantly Black and brown communities, while the white enclaves in the north and northwest of the city have no sensors at all, despite Chicago police data that shows gun crime is spread throughout the city. One country, two systems Arbitrary death by firing squad? Check Arbitrary expropriation of private property? Check Arbitrary internment for one class of society? Recurring theme, but Check

noobermin 5y ago

Stories like this are part and parcel why many people don't trust the police.

toofy 5y ago

> ShotSpotter’s claims about its accuracy have increased, from 80 percent accurate to 90 percent accurate to 97 percent accurate. According to Greene, those numbers aren’t actually calculated by engineers, though. > “Our guarantee was put together by our sales and marketing department, not our engineers,” Greene told a San Francisco court in 2017. “We need to give them [customers] a number … We have to tell them something. … there’s the problem right there. a large number of us have seen, first hand, how a sales and marketing team will happily distort the abilities of something we build. and we’ve all seen first hand how clients will lap that up and then turn around and blame the engineers when it doesn’t live up to what they were led to believe. as his quote says, those numbers aren’t actually calculated by engineers, they’re calculated by the sales and marketing department and he happily pushes this nonsense because “… we have to tell them something.” meanwhile people are dying from this/being prosecuted and the accuracy rates are pulled out of thin air by the marketing/sales department.

at-fates-hands 5y ago
>> and then turn around and blame the engineers when it doesn’t live up to what they were led to believe. Any examples of how shotspotter technology has not lived up to or has failed in what it was designed to do? I'm not really that familiar with the technology, all I know is that if there are shots fired within the vicinity of a shotspotter it registers those shots. Does it have the capability of triangulating exactly where the shot took place? We have them all over Minneapolis and there's several twitter accounts that monitor police activity and I've seen a lot of posts saying: shotspotter was activated at 33rd and Hennepin, police located several bullet casings nearby and are actively searching for the shooter Stuff like that. What's the difference between having 80% and 97% accuracy?
- ashtonkem 5y ago
  > Any examples of how shotspotter technology has not lived up to or has failed in what it was designed to do? You're literally quoting on an article where they moved the location of a sound by over a mile, and reclassified it from a firework to a gunshot. They tried to use this obviously falsified evidence to throw a man in jail, and had to withdraw the evidence once this was pointed out. Sure seems like an example of it not living up to what it was designed to do. Even if it turns out Williams is guilty and he did shoot that man, Shotspotter was just blatantly wrong about what happened and where it happened. Also, 86% of the time it detects a gunshot police find nothing. https://chicago.suntimes.com/2021/5/4/22417660/shotspotter-analysis-macarthur-justice-center-chicago-police-chicago-gun-violence-editorial https://chicago.suntimes.com/2021/5/4/22417660/shotspotter-a... > We have them all over Minneapolis and there's several twitter accounts that monitor police activity and I've seen a lot of posts saying: Ooookay? This is practically the definition of how the plurality of anecdote is not data. > What's the difference between having 80% and 97% accuracy? When it's being used to send armed representatives of the state to arrest and prosecute people? The difference is a lot of people arrested and prosecuted based on faulty evidence.

advael 5y ago

All you need to hear from this article is that no one outside of the company can independently audit the software Any such solution should be dead in the water as a source of meaningful forensic evidence, period

mikewarot 5y ago

Why is this considered "evidence" at all? Raw recordings of sound, with timestamps and locations of sensors, could be evidence, but something output from an AI? Let's also consider that the speed of sound isn't constant, sound reflects off of everything, and can even be ducted by natural causes.

sennight 5y ago

> Rochester police had also refused his multiple requests for them to test his hands and clothing for gunshot residue. GSR tests are total crap and in my experience lean very hard toward the false positive side of things, unsurprisingly nitrates are all over the place. Cops know this, so them not administering a test that can only hurt a suspect's chances with a jury - well the guys on the street certainly weren't out to frame him.

tim333 5y ago

There must be some laws against doing this stuff in the US? In the uk we have 'perverting the course of justice'.