7 ms·

This is tough. Pretty clearly what's happening here is shot spotter is producing a probability distribution about the places where a shot may have been fired, b

by oehpr 5y ago

This is tough. Pretty clearly what's happening here is shot spotter is producing a probability distribution about the places where a shot may have been fired, but they're not showing that information, they're showing a point. That's broken. It's a shame because setting up some microphones around the city to detect a firearm discharge seems like a great idea. The problem is people are overselling what it is doing. Something that jumps out at me in the article is this: Paul Greene, ShotSpotter’s expert witness and an employee of the company, testified at Simmons’ trial that “subsequently he was asked by the Rochester Police Department to essentially search and see if there were more shots fired than ShotSpotter picked up,” according to a civil lawsuit Simmons has filed against the city and the company. Greene found a fifth shot, despite there being no physical evidence at the scene that Simmons had fired. ... Like, what shotspotter is doing is attempting to resolve the origin of a sound wave in a city. The shockwave from a gun blast must reverberate across buildings all over the place, more than likely you're getting reflected waves rather than waves from the source. You're going to get echos and reverb and it's going to be messy and you're going to be imprecise. Would shotspotter's AI misclassify noises? Yah. Of course. If it's like any other classifier it's likely listing out a list of probabilities of what something is and, for the sake of simplicity, just saying it's the highest probability output. It's just a filtering step. If you know a gunshot occurred, you update your priors. Should shotspotter be able to detect the number of gunshots exactly? Maybe? How to differentiate between echos and original? I'm not shocked it can detect the wrong number of shots. Since it misclassifies noises, isn't exactly sure of the origin, and can read echos as additional shots, is it useless? No.

slumdev 5y ago
This is an unreliable system which police and prosecutors misrepresent as reliable. It might not be useless in the strictest sense of the word, but it ought to be taken away from them.
- Frondo 5y ago
  This is the issue: it's unreliable, and its results presented as a matter of fact. This is also the latest I've heard of, in a long line of the criminal justice system presenting what's often closer to fiction as an indisputable fact of someone's guilt. One of the cases that made me incredibly angry was the junk arson science surrounding the conviction and execution of Cameron Todd Willingham in Texas in 2004: https://innocenceproject.org/cameron-todd-willingham-wrongfully-convicted-and-executed-in-texas/ https://innocenceproject.org/cameron-todd-willingham-wrongfu... But I also think of how unreliable eyewitness testimony is, and yet people's recollections of events (when we know, and have known for so long, that memory is plastic!), sometimes years later, are treated as fact: https://www.ncsc.org/trends/monthly-trends-articles/2017/the-trouble-with-eyewitness-identification-testimony-in-criminal-cases#:~:text=Research%20has%20found%20that%20eyewitness-identification%20testimony%20can%20be%20very%20unreliable.&text=Although%20witnesses%20can%20often%20be,most%20unreliable%20forms%20of%20evidence https://www.ncsc.org/trends/monthly-trends-articles/2017/the.... I don't know how to solve this, since the "proven" part of "innocent until proven guilty" seems like it's built on a house of cards, and the only people with a direct incentive to change it are suspects (i.e. the people that the criminal justice system has already decided could've done it, and how can you trust a criminal's motivations? eye roll)
- meowface 5y ago
  Restrict its use to the original intention: detecting potential events police might want to immediately respond to. It (and any other gunfire locator) seems like it'd probably be an extremely useful and publicly beneficial technology even if it had a low true positive rate. The problem is allowing it to be used as evidence in criminal cases. That's the part that should be taken away.
  - tomschlick 5y ago
    This exactly. I see Shotspotter and tech like facial recognition / license plate readers as the same. Its a lead, a clue for detectives to follow up on and validate / verify. It shouldn't be evidence on its own. Just a source that's slightly more reliable than a human witness.
    - meowface 5y ago
      The arguments for facial recognition and license plate readers are much more complex. In the case of a gunfire detector, you're specifically looking for (supposed) activity that indicates the most serious possible crime might have occurred seconds ago. Facial recognition and license plate readers are used to automatically track individual humans wherever they go so that they can be quickly arrested if they're on a to-arrest list. I'm not entirely sure how I feel about that, even if it's all taking place in public, open areas, but it's a totally different category from just knowing when and where a gun may have been fired. It's real-time crime detection vs. alleged criminal detection.
      - tomschlick 5y ago
        Thats very true. I also have reservations around facial recognition / ALPR systems if they are saving data of people who are not currently "flagged" related to a crime. I can see the usefulness to find stolen cars, or track down murder suspects but only after they are entered as a suspect. There shouldn't be a historical database of previous movements. There should also be states/federal guidelines around who can be entered.
- formerly_proven 5y ago
  This seems to be intentional: > Rather than defend ShotSpotter’s technology and its employees' actions in a Frye hearing, the prosecutors withdrew all ShotSpotter evidence against Williams. Ducking the test and withdrawing the evidence means prosecutors can continue to claim that this stuff hasn't been deemed unreliable by experts.
  - slumdev 5y ago
    A quirk of our legal system that allows dishonest people to continue with impunity.
    - avs733 5y ago
      which courts could easily fix...but don't.
- avs733 5y ago
  > shot spotter is producing a probability distribution about the places where a shot may have been fired, but they're not showing that information, they're showing a point. That's broken. >This is an unreliable system which police and prosecutors misrepresent as reliable. I would argue in american policing, that broken is the point. People (cops, judges, juries, the public) don't engage with anything criminal justice related as probabilistic. If they showed the probabilities, they wouldn't have a product specifically because it would not be able presented as reliable. It's product market fit, like the parallel citizen thread, that serves malicious and anti-social goals. But it is profitable.
  - kevin_thibedeau 5y ago
    It's the same deal with breathalyzers, which don't measure blood alcohol and cannot estimate it to the levels of precision the manufacturers want everyone to believe.
    - avs733 5y ago
      and a pattern of withdrawing the evidence when the results are challenged so as to avoid setting precedent.
    - giantg2 5y ago
      The "evidence" can't be reexamined either. The good states require a blood test. At least it's more accurate and can be examined by the defense.
modriano 5y ago
Yeah, it's a really useful tool. There are a lot of gunshots in Chicago (last year there were over 4000 incidents (in Chicago) where someone was shot, and there are far more gunshots where no one is hit), and in many of those shootings, no one calls 911. If someone is hit and rendered unable to call 911 or get help, if no one calls 911, that shooting victims is likely to become a homicide victim. ShotSpotter gets someone to check the area out, and it has saved a number of people from bleeding out. I don't think it should be used as evidence of the location of a shooting (often shootings are only discovered when the victim shows up at a hospital), as locations are triangulated based on the time different sensors detect a gunshot and the distance sound travels is impacted by building geometry and materials. But it's a very useful tool that saves lives. Regarding the locations of ShotSpotter coverage, as the pricing model is proportional to area covered (there's a physical network that needs to be maintained and contacts with owners of buildings with sensors that need to be paid), so sensors are put in the areas with lots of shootings. If you want to know why there is such extreme racial homogeneity in the most violent, least well resourced parts of Chicago, I highly recommend the book "The Color of Law" [0] on redlining. [0] https://www.epi.org/publication/the-color-of-law-a-forgotten-history-of-how-our-government-segregated-america/ https://www.epi.org/publication/the-color-of-law-a-forgotten...
- loteck 5y ago
  Do you have any citations for your claims about the Chicago system being responsible for saving a number of people? I'm looking for reading on that topic. Thanks ahead of time.
  - modriano 5y ago
    None that are public (that I'm aware of), but it's just a matter of time. We just got ShotSpotter alerts put up on the Chicago data portal. But to make it less difficult to believe, I'll point you too LEMART training [1], an emergency first aid course that many patrol officers elect to take, and that many officers carry a first aid kit [2]. Considering ShotSpotter alerts officers (many of which are capable of administering emergency first aid) to shootings (many of which have no corresponding 911 call for service), it's not too hard to accept the weak claim that a nonzero number of lives have been saved. I won't speculate on the number as I haven't done a rigorous analysis, but I know of a lot of shootings where the responding officer was notified by ShotSpotter, administered first aid (normally a tourniquet), and the victim survived. Edit: just found a public number for the number of officers who have received LEMART training. Over 6000 officers as of 1/21/2020 [3]. For reference, there are about 13k sworn CPD officers. [0] https://data.cityofchicago.org/Public-Safety/Violence-Reduction-Shotspotter-Alerts/3h7q-7mdb https://data.cityofchicago.org/Public-Safety/Violence-Reduct... [1] https://www.chicagocopa.org/wp-content/uploads/2018/04/Policy-Report-re-First-Aid-Policy-Training-Equipment.pdf https://www.chicagocopa.org/wp-content/uploads/2018/04/Polic... [2] https://directives.crimeisdown.com/diff/efaff1a7e94a8ad642c793993e0d6701c1d276fd/directives/data/a7a57b38-1488e40c-5eb14-88e6-c8b737ad46ecc09a.html https://directives.crimeisdown.com/diff/efaff1a7e94a8ad642c7... [3] https://twitter.com/chicago_police/status/1219747858469003265 https://twitter.com/chicago_police/status/121974785846900326...
loteck 5y ago
It's a shame because setting up some microphones around the city to detect a firearm discharge seems like a great idea. It's an idea, yes, but a *great* idea? There are so many technical, social and civil rights issues wrapped up in this idea on its face, I think its performance history should be part of whether it gets to enjoy the label of being a "great idea." Since it misclassifies noises, isn't exactly sure of the origin, and can read echos as additional shots, is it useless? Furthermore, since it can and does record human voices speaking, and creates permanent records into the private hands of a for-profit company that is allowed to sell that data to 3rd parties, does it actually cross a line from being useless to being harmful? It might.
- oehpr 5y ago
  Right... but I don't classify its performance of "saying 5 gunshots when their was 4" a failure. If it can say "I heard a gunshot, sounded like it came from around here." and a gunshot occurred somewhere around there, with only a few false positives. That's fantastic. In my view, that's "great". It's just not iron clad evidence, prosecutors are overstating its statistical power. That needs to be checked. Furthermore, since it can and does record human voices speaking, and creates permanent records into the private hands of a for-profit company that is allowed to sell that data to 3rd parties, does it actually cross a line from being useless to being harmful? It might. Yah absolutely. But the thing is gunshots are pretty loud noises compared to human speech. There's no reason to think a microphone needs to be sensitive enough to capture human speech to capture a gunshot, nor placed in a location that is well suited for capturing human speech, nor would a place that IS well suited for capturing human speech a good location for capturing gunshots. If I was placing a microphone to capture the sound of gunshots, I'd be putting it on the tops of streetlights, telephone polls, and the corners of buildings. You're not going to be hearing people from there. So if we see that behavior of ShotSpotter sticking microphones in inappropriate places for public surveillance purposes, then I think they should be taken to task for it.
  - deleted 5y ago
    [deleted]
  - 5y ago
Imnimo 5y ago
I think the problem isn't the technology or the idea, it's the business model and the associated incentives. ShotSpotter's customers are the police, and the objective of police is to secure convictions, so ShotSpotter is incentivized to help them do that, not to produce true and accurate predictions. Exonerating a defendant isn't going to win them more business.
- spywaregorilla 5y ago
  > the objective of police is to secure convictions I mean... that's the problem point it would seem
- jfrankamp 5y ago
  Right. Shotspotter or the generic equiv should be purchased by the city, not the police. The data (raw and interpreted) should be managed in the public, as its a public interest to know about firearms (including the police's) being discharged.
  - dragonwriter 5y ago
    > Shotspotter or the generic equiv should be purchased by the city, not the police. Generally, the police are part of the city, and the sole and deliberately centralized city agency with a mandate that makes something like ShotSpotter relevant. So, of course, if it is bought “by the city”, it is through and under control of the police department.
    - jfrankamp 5y ago
      I'm arguing though that due to the conflict of interest, it should be at least one level up IRT purchasing, access to interpretive/raw data, and decision making on which of that data they want to share etc. Remove funding and responsibility from the police to run this system/vendor, and move it into the "office of public data accountability" e.g. which can serve its data equally to the public, watchdog groups and the police.
deleted 5y ago
[deleted]
cryptonector 5y ago
Overselling + altering reports after the fact to fit a prosecution's preferred theory of the events.
BoorishBears 5y ago
How is this tough?! The system detected a loud noise as fireworks... that should be the end of it! Let the cards fall where they may, if cops go out they go out, they go out, otherwise that's that. You don't get to go back days later and reclassify it as a gunshot based on ambiguity inherent to your product, and you don't get to go back and move the origin. This is not tough at all, it literally should work exactly like it works today, and no one is allowed to change the initial call manually. After all, who is helped by retroactively marking it as a gunshot but a DA? - If the concern is building up statistics or something, ShotSpotter is still recording that it heard a sound. Nothing is stopping an out of band note that says "we believe this recording was tied to so and so incident at so and so location" The key difference though is that ShotSpotter is no long claiming to have recorded that incident directly. Which matters when people's freedom is at stake.

by oehpr 5y ago

slumdev 5y ago

This is an unreliable system which police and prosecutors misrepresent as reliable. It might not be useless in the strictest sense of the word, but it ought to be taken away from them.

Frondo 5y ago
This is the issue: it's unreliable, and its results presented as a matter of fact. This is also the latest I've heard of, in a long line of the criminal justice system presenting what's often closer to fiction as an indisputable fact of someone's guilt. One of the cases that made me incredibly angry was the junk arson science surrounding the conviction and execution of Cameron Todd Willingham in Texas in 2004: https://innocenceproject.org/cameron-todd-willingham-wrongfully-convicted-and-executed-in-texas/ https://innocenceproject.org/cameron-todd-willingham-wrongfu... But I also think of how unreliable eyewitness testimony is, and yet people's recollections of events (when we know, and have known for so long, that memory is plastic!), sometimes years later, are treated as fact: https://www.ncsc.org/trends/monthly-trends-articles/2017/the-trouble-with-eyewitness-identification-testimony-in-criminal-cases#:~:text=Research%20has%20found%20that%20eyewitness-identification%20testimony%20can%20be%20very%20unreliable.&text=Although%20witnesses%20can%20often%20be,most%20unreliable%20forms%20of%20evidence https://www.ncsc.org/trends/monthly-trends-articles/2017/the.... I don't know how to solve this, since the "proven" part of "innocent until proven guilty" seems like it's built on a house of cards, and the only people with a direct incentive to change it are suspects (i.e. the people that the criminal justice system has already decided could've done it, and how can you trust a criminal's motivations? eye roll)
meowface 5y ago
Restrict its use to the original intention: detecting potential events police might want to immediately respond to. It (and any other gunfire locator) seems like it'd probably be an extremely useful and publicly beneficial technology even if it had a low true positive rate. The problem is allowing it to be used as evidence in criminal cases. That's the part that should be taken away.
- tomschlick 5y ago
  This exactly. I see Shotspotter and tech like facial recognition / license plate readers as the same. Its a lead, a clue for detectives to follow up on and validate / verify. It shouldn't be evidence on its own. Just a source that's slightly more reliable than a human witness.
  - meowface 5y ago
    The arguments for facial recognition and license plate readers are much more complex. In the case of a gunfire detector, you're specifically looking for (supposed) activity that indicates the most serious possible crime might have occurred seconds ago. Facial recognition and license plate readers are used to automatically track individual humans wherever they go so that they can be quickly arrested if they're on a to-arrest list. I'm not entirely sure how I feel about that, even if it's all taking place in public, open areas, but it's a totally different category from just knowing when and where a gun may have been fired. It's real-time crime detection vs. alleged criminal detection.
    - tomschlick 5y ago
      Thats very true. I also have reservations around facial recognition / ALPR systems if they are saving data of people who are not currently "flagged" related to a crime. I can see the usefulness to find stolen cars, or track down murder suspects but only after they are entered as a suspect. There shouldn't be a historical database of previous movements. There should also be states/federal guidelines around who can be entered.
formerly_proven 5y ago
This seems to be intentional: > Rather than defend ShotSpotter’s technology and its employees' actions in a Frye hearing, the prosecutors withdrew all ShotSpotter evidence against Williams. Ducking the test and withdrawing the evidence means prosecutors can continue to claim that this stuff hasn't been deemed unreliable by experts.
- slumdev 5y ago
  A quirk of our legal system that allows dishonest people to continue with impunity.
  - avs733 5y ago
    which courts could easily fix...but don't.
avs733 5y ago
> shot spotter is producing a probability distribution about the places where a shot may have been fired, but they're not showing that information, they're showing a point. That's broken. >This is an unreliable system which police and prosecutors misrepresent as reliable. I would argue in american policing, that broken is the point. People (cops, judges, juries, the public) don't engage with anything criminal justice related as probabilistic. If they showed the probabilities, they wouldn't have a product specifically because it would not be able presented as reliable. It's product market fit, like the parallel citizen thread, that serves malicious and anti-social goals. But it is profitable.
- kevin_thibedeau 5y ago
  It's the same deal with breathalyzers, which don't measure blood alcohol and cannot estimate it to the levels of precision the manufacturers want everyone to believe.
  - avs733 5y ago
    and a pattern of withdrawing the evidence when the results are challenged so as to avoid setting precedent.
  - giantg2 5y ago
    The "evidence" can't be reexamined either. The good states require a blood test. At least it's more accurate and can be examined by the defense.

modriano 5y ago

Yeah, it's a really useful tool. There are a lot of gunshots in Chicago (last year there were over 4000 incidents (in Chicago) where someone was shot, and there are far more gunshots where no one is hit), and in many of those shootings, no one calls 911. If someone is hit and rendered unable to call 911 or get help, if no one calls 911, that shooting victims is likely to become a homicide victim. ShotSpotter gets someone to check the area out, and it has saved a number of people from bleeding out. I don't think it should be used as evidence of the location of a shooting (often shootings are only discovered when the victim shows up at a hospital), as locations are triangulated based on the time different sensors detect a gunshot and the distance sound travels is impacted by building geometry and materials. But it's a very useful tool that saves lives. Regarding the locations of ShotSpotter coverage, as the pricing model is proportional to area covered (there's a physical network that needs to be maintained and contacts with owners of buildings with sensors that need to be paid), so sensors are put in the areas with lots of shootings. If you want to know why there is such extreme racial homogeneity in the most violent, least well resourced parts of Chicago, I highly recommend the book "The Color of Law" [0] on redlining. [0] https://www.epi.org/publication/the-color-of-law-a-forgotten-history-of-how-our-government-segregated-america/ https://www.epi.org/publication/the-color-of-law-a-forgotten...

loteck 5y ago
Do you have any citations for your claims about the Chicago system being responsible for saving a number of people? I'm looking for reading on that topic. Thanks ahead of time.
- modriano 5y ago
  None that are public (that I'm aware of), but it's just a matter of time. We just got ShotSpotter alerts put up on the Chicago data portal. But to make it less difficult to believe, I'll point you too LEMART training [1], an emergency first aid course that many patrol officers elect to take, and that many officers carry a first aid kit [2]. Considering ShotSpotter alerts officers (many of which are capable of administering emergency first aid) to shootings (many of which have no corresponding 911 call for service), it's not too hard to accept the weak claim that a nonzero number of lives have been saved. I won't speculate on the number as I haven't done a rigorous analysis, but I know of a lot of shootings where the responding officer was notified by ShotSpotter, administered first aid (normally a tourniquet), and the victim survived. Edit: just found a public number for the number of officers who have received LEMART training. Over 6000 officers as of 1/21/2020 [3]. For reference, there are about 13k sworn CPD officers. [0] https://data.cityofchicago.org/Public-Safety/Violence-Reduction-Shotspotter-Alerts/3h7q-7mdb https://data.cityofchicago.org/Public-Safety/Violence-Reduct... [1] https://www.chicagocopa.org/wp-content/uploads/2018/04/Policy-Report-re-First-Aid-Policy-Training-Equipment.pdf https://www.chicagocopa.org/wp-content/uploads/2018/04/Polic... [2] https://directives.crimeisdown.com/diff/efaff1a7e94a8ad642c793993e0d6701c1d276fd/directives/data/a7a57b38-1488e40c-5eb14-88e6-c8b737ad46ecc09a.html https://directives.crimeisdown.com/diff/efaff1a7e94a8ad642c7... [3] https://twitter.com/chicago_police/status/1219747858469003265 https://twitter.com/chicago_police/status/121974785846900326...

loteck 5y ago

It's a shame because setting up some microphones around the city to detect a firearm discharge seems like a great idea. It's an idea, yes, but a *great* idea? There are so many technical, social and civil rights issues wrapped up in this idea on its face, I think its performance history should be part of whether it gets to enjoy the label of being a "great idea." Since it misclassifies noises, isn't exactly sure of the origin, and can read echos as additional shots, is it useless? Furthermore, since it can and does record human voices speaking, and creates permanent records into the private hands of a for-profit company that is allowed to sell that data to 3rd parties, does it actually cross a line from being useless to being harmful? It might.

oehpr 5y ago
Right... but I don't classify its performance of "saying 5 gunshots when their was 4" a failure. If it can say "I heard a gunshot, sounded like it came from around here." and a gunshot occurred somewhere around there, with only a few false positives. That's fantastic. In my view, that's "great". It's just not iron clad evidence, prosecutors are overstating its statistical power. That needs to be checked. Furthermore, since it can and does record human voices speaking, and creates permanent records into the private hands of a for-profit company that is allowed to sell that data to 3rd parties, does it actually cross a line from being useless to being harmful? It might. Yah absolutely. But the thing is gunshots are pretty loud noises compared to human speech. There's no reason to think a microphone needs to be sensitive enough to capture human speech to capture a gunshot, nor placed in a location that is well suited for capturing human speech, nor would a place that IS well suited for capturing human speech a good location for capturing gunshots. If I was placing a microphone to capture the sound of gunshots, I'd be putting it on the tops of streetlights, telephone polls, and the corners of buildings. You're not going to be hearing people from there. So if we see that behavior of ShotSpotter sticking microphones in inappropriate places for public surveillance purposes, then I think they should be taken to task for it.
- deleted 5y ago
  [deleted]
- 5y ago

Imnimo 5y ago

I think the problem isn't the technology or the idea, it's the business model and the associated incentives. ShotSpotter's customers are the police, and the objective of police is to secure convictions, so ShotSpotter is incentivized to help them do that, not to produce true and accurate predictions. Exonerating a defendant isn't going to win them more business.

spywaregorilla 5y ago
> the objective of police is to secure convictions I mean... that's the problem point it would seem
jfrankamp 5y ago
Right. Shotspotter or the generic equiv should be purchased by the city, not the police. The data (raw and interpreted) should be managed in the public, as its a public interest to know about firearms (including the police's) being discharged.
- dragonwriter 5y ago
  > Shotspotter or the generic equiv should be purchased by the city, not the police. Generally, the police are part of the city, and the sole and deliberately centralized city agency with a mandate that makes something like ShotSpotter relevant. So, of course, if it is bought “by the city”, it is through and under control of the police department.
  - jfrankamp 5y ago
    I'm arguing though that due to the conflict of interest, it should be at least one level up IRT purchasing, access to interpretive/raw data, and decision making on which of that data they want to share etc. Remove funding and responsibility from the police to run this system/vendor, and move it into the "office of public data accountability" e.g. which can serve its data equally to the public, watchdog groups and the police.

deleted 5y ago

[deleted]

cryptonector 5y ago

Overselling + altering reports after the fact to fit a prosecution's preferred theory of the events.

BoorishBears 5y ago

How is this tough?! The system detected a loud noise as fireworks... that should be the end of it! Let the cards fall where they may, if cops go out they go out, they go out, otherwise that's that. You don't get to go back days later and reclassify it as a gunshot based on ambiguity inherent to your product, and you don't get to go back and move the origin. This is not tough at all, it literally should work exactly like it works today, and no one is allowed to change the initial call manually. After all, who is helped by retroactively marking it as a gunshot but a DA? - If the concern is building up statistics or something, ShotSpotter is still recording that it heard a sound. Nothing is stopping an out of band note that says "we believe this recording was tied to so and so incident at so and so location" The key difference though is that ShotSpotter is no long claiming to have recorded that incident directly. Which matters when people's freedom is at stake.