6 ms·

Would results of discovery be public record? If so, I could also see: 3) They don't want their "secret sauce" made publicly available. That would both open th

Would results of discovery be public record? If so, I could also see: 3) They don't want their "secret sauce" made publicly available. That would both open them up to commercial competition, and make their product less reliable in court, as anti-Shotspotter experts would start figuring out how to effectively argue against the product's methodology. In general I think technology like this should have to be fully transparent if it is used as the basis for public entities' decisions, but so far the arguments against that practice seem to have won out. For example, good luck getting source code or design documents to contest an automated red light or speeding ticket.

tomrod 4y ago
For real. I wonder how active phone microphones and geolocation are generally -- I know there are a host of location tracking (marketing) companies that use app geolocated data as their sources.
solardev 4y ago
Please report to the nearest processing center for your attempted libel against the police-industrial complex.
flaviut 4y ago
This is a reasonable argument, but I'm not sure you're aware of allegations of this company altering evidence to better fit a police narrative: https://www.vice.com/en/article/qj8xbq/police-are-telling-shotspotter-to-alter-evidence-from-gunshot-detecting-ai https://www.vice.com/en/article/qj8xbq/police-are-telling-sh... I sincerely doubt there's any secret sauce to protect when they have "analysts" sitting around regularly "correcting" evidence.
- pseudo0 4y ago
  I agree that there is definitely moral hazard when it comes to after the fact "corrections", particularly if the analyst knows the outcome desired by police. The company's track record looks pretty poor in this respect. In general though, I think there are legitimate reasons for manual analysis. Any machine learning approach to a problem like this is going to have to balance false positives and false negatives, and there is necessarily going to be a somewhat arbitrary cutoff. Detecting bang-like noises in a large city is likely going to have a pretty conservative cutoff to avoid DDOSing the police with calls. For example say the company sets their cutoff for automatic reporting at 90% confidence, but when the police ask them to review a specific time period, it turns out there was a shot detected with 89% confidence, and manual analysis indicates a false negative. That is probably still useful information, and it would definitely be useful to include this data point with the correct classification in future iterations of the model.
oneoff786 4y ago
There’s no way they have a legitimate secret sauce though… right? This doesn’t seem like a hard problem to do. Maybe some of this https://en.m.wikipedia.org/wiki/Independent_component_analysis https://en.m.wikipedia.org/wiki/Independent_component_analys... Plus basic triangulation logic synced across a bunch of microphones?
- kencausey 4y ago
  There is also the matter of correctly identifying gunshots.
  - anigbrowl 4y ago
    They get lots of false positives, from band practice to backfiring vehicle engines and (of course) fireworks.
- HWR_14 4y ago
  It's phenomenally hard. Echos, indirect routing (bouncing off buildings), sound waves changing speed as they go through various materials (like a lens). It's a very hard problem. Heck, a truly calibrated surround sound system requires using a microphone to measure the volume in place.
  - matt_heimer 4y ago
    So can't you just do that? Install the system and then drive a calibration car that emits sounds and logs gps coordinates?
    - beckingz 4y ago
      Map every city in the US and world? Now that's a lot of work.
      - sbierwagen 4y ago
        Google was collecting point cloud data of US cities 15 years ago, I assume some GIS vendor somewhere is selling a copycat dataset to self driving car startups.
gorgoiler 4y ago
If your business is to help uphold the public rule of law it’s a stretch to want to keep that process a business secret. You can’t make rulers while also trying to keep the definition of an inch secret.
HWR_14 4y ago
I'm sensitive to the idea their methodology could be used to create a competing product, and think the court could get the experts to agree to event more stringent than normal NDAs to examine it. But I think if a computer system is going to be used to convict someone, experts effectively arguing against it's methodology is good. In fact, I think experts should have access to various machines that could convict someone. I don't really care about a red light camera (as the image shows the license plate and the red light at the same time), but there are tons of other technical solutions that need to be examined.
anigbrowl 4y ago
They could ask for the information to be under seal to protect their commercial interests. The actual technology is not all that complex. Transient detection and triangulation by decorrelation from a microphone (edit: not microwave!) array. You could probably prototype it in a week if you leaned on an established FFT library. Obviously they would not want to reveal their source code and so on, but if you know digital audio you know this isn't that hard. I think what they really want to conceal is not the technology but its use in operation. I don't know what Chicago pays for the service, but Sacramento CA spends about $1m a year on it and Sac is a fairly small city. https://www.sacbee.com/news/local/article231997777.html https://www.sacbee.com/news/local/article231997777.html
- SoftTalker 4y ago
  Chicago Aldermen are no doubt getting a generous kickback on whatever they are paying.
- toomuchtodo 4y ago
  While true the tech is not complex, I believe the discrepancy is "how accurate is it and is the data being manipulated to fit a crime" versus "the data is accurate and leading to fair justice outcomes." Wouldn't "red teaming" ShotSpotter solve for this to see how accurate they are? A team would perform controlled firings in known locations under various auditory scenes with the city's approval, and then request the data from ShotSpotter to see how accurate they are (without revealing you are performing an exercise). ShotSpotter then doesn't have to reveal their methodology, as their data provided from a blind test will reveal the accuracy regardless of how they arrive at conclusions. The system is either accurate or it isn't (with a measured level of confidence), and this can be tested for without ShotSpotter's consent (as long as they continue to provide data to the customer per their contract).
  - cortesoft 4y ago
    That would tell you if it correctly detected gunshots... but it wouldn't tell you if it falsely reported certain sounds as gunshots, if it works in all weather, etc.
thathndude 4y ago
If the real concern was secrecy, there are protective orders and other techniques that could go pretty far to assure this (but nothing is impossible). I think the most reasonable inference is that they are outright trying to hide. And probably because the truth is totally damning.
faeriechangling 4y ago
I think this company is probably more successful the less potential buyers know about it, the product sounds like total trash to me.
lamontcg 4y ago
> and make their product less reliable in court, as anti-Shotspotter experts would start figuring out how to effectively argue against the product's methodology. If their methodology has weaknesses which allow those arguments, then its kind of a travesty of justice not to allow them.

tomrod 4y ago

For real. I wonder how active phone microphones and geolocation are generally -- I know there are a host of location tracking (marketing) companies that use app geolocated data as their sources.

solardev 4y ago

Please report to the nearest processing center for your attempted libel against the police-industrial complex.

flaviut 4y ago

This is a reasonable argument, but I'm not sure you're aware of allegations of this company altering evidence to better fit a police narrative: https://www.vice.com/en/article/qj8xbq/police-are-telling-shotspotter-to-alter-evidence-from-gunshot-detecting-ai https://www.vice.com/en/article/qj8xbq/police-are-telling-sh... I sincerely doubt there's any secret sauce to protect when they have "analysts" sitting around regularly "correcting" evidence.

pseudo0 4y ago
I agree that there is definitely moral hazard when it comes to after the fact "corrections", particularly if the analyst knows the outcome desired by police. The company's track record looks pretty poor in this respect. In general though, I think there are legitimate reasons for manual analysis. Any machine learning approach to a problem like this is going to have to balance false positives and false negatives, and there is necessarily going to be a somewhat arbitrary cutoff. Detecting bang-like noises in a large city is likely going to have a pretty conservative cutoff to avoid DDOSing the police with calls. For example say the company sets their cutoff for automatic reporting at 90% confidence, but when the police ask them to review a specific time period, it turns out there was a shot detected with 89% confidence, and manual analysis indicates a false negative. That is probably still useful information, and it would definitely be useful to include this data point with the correct classification in future iterations of the model.

oneoff786 4y ago

There’s no way they have a legitimate secret sauce though… right? This doesn’t seem like a hard problem to do. Maybe some of this https://en.m.wikipedia.org/wiki/Independent_component_analysis https://en.m.wikipedia.org/wiki/Independent_component_analys... Plus basic triangulation logic synced across a bunch of microphones?

kencausey 4y ago
There is also the matter of correctly identifying gunshots.
- anigbrowl 4y ago
  They get lots of false positives, from band practice to backfiring vehicle engines and (of course) fireworks.
HWR_14 4y ago
It's phenomenally hard. Echos, indirect routing (bouncing off buildings), sound waves changing speed as they go through various materials (like a lens). It's a very hard problem. Heck, a truly calibrated surround sound system requires using a microphone to measure the volume in place.
- matt_heimer 4y ago
  So can't you just do that? Install the system and then drive a calibration car that emits sounds and logs gps coordinates?
  - beckingz 4y ago
    Map every city in the US and world? Now that's a lot of work.
    - sbierwagen 4y ago
      Google was collecting point cloud data of US cities 15 years ago, I assume some GIS vendor somewhere is selling a copycat dataset to self driving car startups.

gorgoiler 4y ago

If your business is to help uphold the public rule of law it’s a stretch to want to keep that process a business secret. You can’t make rulers while also trying to keep the definition of an inch secret.

HWR_14 4y ago

I'm sensitive to the idea their methodology could be used to create a competing product, and think the court could get the experts to agree to event more stringent than normal NDAs to examine it. But I think if a computer system is going to be used to convict someone, experts effectively arguing against it's methodology is good. In fact, I think experts should have access to various machines that could convict someone. I don't really care about a red light camera (as the image shows the license plate and the red light at the same time), but there are tons of other technical solutions that need to be examined.

anigbrowl 4y ago

They could ask for the information to be under seal to protect their commercial interests. The actual technology is not all that complex. Transient detection and triangulation by decorrelation from a microphone (edit: not microwave!) array. You could probably prototype it in a week if you leaned on an established FFT library. Obviously they would not want to reveal their source code and so on, but if you know digital audio you know this isn't that hard. I think what they really want to conceal is not the technology but its use in operation. I don't know what Chicago pays for the service, but Sacramento CA spends about $1m a year on it and Sac is a fairly small city. https://www.sacbee.com/news/local/article231997777.html https://www.sacbee.com/news/local/article231997777.html

SoftTalker 4y ago
Chicago Aldermen are no doubt getting a generous kickback on whatever they are paying.
toomuchtodo 4y ago
While true the tech is not complex, I believe the discrepancy is "how accurate is it and is the data being manipulated to fit a crime" versus "the data is accurate and leading to fair justice outcomes." Wouldn't "red teaming" ShotSpotter solve for this to see how accurate they are? A team would perform controlled firings in known locations under various auditory scenes with the city's approval, and then request the data from ShotSpotter to see how accurate they are (without revealing you are performing an exercise). ShotSpotter then doesn't have to reveal their methodology, as their data provided from a blind test will reveal the accuracy regardless of how they arrive at conclusions. The system is either accurate or it isn't (with a measured level of confidence), and this can be tested for without ShotSpotter's consent (as long as they continue to provide data to the customer per their contract).
- cortesoft 4y ago
  That would tell you if it correctly detected gunshots... but it wouldn't tell you if it falsely reported certain sounds as gunshots, if it works in all weather, etc.

thathndude 4y ago

If the real concern was secrecy, there are protective orders and other techniques that could go pretty far to assure this (but nothing is impossible). I think the most reasonable inference is that they are outright trying to hide. And probably because the truth is totally damning.

faeriechangling 4y ago

I think this company is probably more successful the less potential buyers know about it, the product sounds like total trash to me.

lamontcg 4y ago

> and make their product less reliable in court, as anti-Shotspotter experts would start figuring out how to effectively argue against the product's methodology. If their methodology has weaknesses which allow those arguments, then its kind of a travesty of justice not to allow them.