12 ms·

AI Is Getting Better at Detecting Handguns

29 points by mcspecter 9y ago

bajsejohannes 9y ago
> what if police officers were equipped with body cameras that could alert them the instant they come into view of a handgun, or some other weapon? And then > Overall, the optimized model achieved 84% accuracy, with zero false negatives [...] and 57 false positives. So in the what-if scenario above, 16% of cases would falsely escalate a situation dramatically? Seems like a horrible idea. And "really good" seems like a stretch.
- mgarfias 9y ago
  What happens when it pushes the panic button when a firearm is present, but carried without threat, legally?
- mgarfias 9y ago
  What happens when it pushes the panic button when a firearm is present, but carried without threat, legally?
- codazoda 9y ago
  There are also a lot of legal gun owners who have a gun but are not a threat.
  - umlaut 9y ago
    That's true in the US, but not in many other places (the UK for example).
  - cityhall 9y ago
    They seem to be spotting visible guns, not concealed ones, and the focus is on guns being brandished instead of holstered. This is preliminary work but it wouldn't be useful to have an alarm go off every time another police officer or security guard comes into view with a gun on his hip.
- artursapek 9y ago
  I guess you'd have to compare the failure rate to that of humans, similar to automated driving.
- bdowling 9y ago
  The usefulness of a system like this would be where the police have a suspect cornered with their weapons drawn, but the suspect refuses to comply with instructions. If the suspect makes a suspicious move (e.g. as if he is reaching to pull a gun from a back pocket) the police will shoot immediately, since they would not have time to react if they waited to see if he was pulling a gun or not. Right now, the human system provides a 100% false positive rate. A weapon system that incorporates this detector would allow the police to pull the trigger when the suspect makes the suspicious move, but delay the firing of the weapon until the system detects a gun. Even with a 16% false positive rate it would save lives. That said, I don't think police would want it if it might increase the risk to their lives.
  - Phlarp 9y ago
    Adding any kind of electronic firing control mechanism to a service pistol is a non-starter for a bunch of reasons: MTBF: Even small guns produce recoil, which over time will damage or degrade any electronics built into the firing mechanism. Security: if a "computer" of whatever complexity is controlling the firing of the gun it presents surface area both for attacks and bugs. Rank and file acceptance: No officer will want to go into the field with a service weapon fitted with "safety restrictions" or "casualty reduction mechanisms" other than the trigger safety.
    - emiliobumachar 9y ago
      I agree with everything you said, except MTBF. The training guns could be separate. Surely the "on-duty" guns are not being fired often enough that this will be an issue at all... right? Right??
      - Phlarp 9y ago
        #1 feeds into #3-- How confident are your rank and file going to be when the "training" guns fail regularly. Also there is something to be said for training with the exact same weapon you plan to use in the field.
- burkaman 9y ago
  That number should be compared to the rate human officers unnecessarily escalate a situation because they think there's a gun. I don't know that number but it could easily be higher than 16%.
  - parsnips 9y ago
    It could easily be lower than 16% too. Much lower.
    - burkaman 9y ago
      Right, I'm saying I don't know the number, but this algorithm shouldn't be judged against a theoretical perfect officer, because that's not reality.
  - wayn3 9y ago
    It should really be compared to the number of instances human officer's act as if there could be a gun " just in case ". Like in a traffic stop. If officer's don't have to potentially gun someone down because they made a hectic movement 84% of the time, that's a pretty big win. Imagine 84% of " black guy mysteriously died" actually led to convictions of officers because their ai thing told them "definitely no gun, bro"
    - dragonwriter 9y ago
      > Imagine 84% of " black guy mysteriously died" actually led to convictions of officers because their ai thing told them "definitely no gun, bro" I imagine that, like body cameras, the AI thing will, in those events, conveniently be deactivated, malfunctioning, misplaced, or have it's records lost.
      - wayn3 9y ago
        probably. but you can only pull that stunt so many times.
        9y ago
- jellicle 9y ago
  That sort of accuracy seems like about three orders of magnitude off from what would be useful in the field. Unless you just basically want a drug-sniffing dog, for guns. That is, something to "scienceify" something that would otherwise be problematic (searching this car, or shooting that suspect). "Hey judge, the DOG said it, it must be true!" "Hey judge, the COMPUTER said it, it must be true!"
- RUG3Y 9y ago
  Beside that, the camera would have to view the subject from the correct angle, so, from the back, most likely position if they're carrying concealed.
- tbabb 9y ago
  16% false positive is hilariously bad, especially given how rare actual handguns will be. This headline shouldn't be tolerated here.
  - jaclaz 9y ago
    On the other hand missing completely it in 3 out of 30 tests (samples with a number of selected "optimizations") is a "terrible" 10% is awfully worse (in an alarm system). >For the experiments, we selected 30 scenes from the previously used videos with the next requirements. Each scene is made up of at least 5 frames, filmed in a fixed scenario, i.e., in the same place, and the pistols are clearly visible to a human viewer. ... The model successfully detects the pistol in 27 scenes with an average time interval AATpI=0.2 seconds, which is good enough for an alarm system. The detector fails to detect pistols only in three scenes. I am probably particularly picky but (from the paper): >This section analyzes the performance of our best detection model on seven public videos, with low qualities, downloaded from youtube. Six of them are pieces of well known films from the 90s, James Bon: The World is Not Enough (video 1), 3 pieces from Pulp Fiction (video 2, 3 and 4), Impossible Mission: Rogue Nation (video 5) and Mister Bin (video 6). The seventh video is a long pistol threatening video (video 7). ... if you cannot spell Bond or Mister Bean, and you classify Mission Impossible: Rogue Nation as a 90's movie (calling it Impossible Mission) you don't look like having been particularly accurate when writing the article, if the same accurateness is in the actual numbers, there is a problem. Anyway the "final" sample (30 scenes) is so small that it makes very little sense to derive anything from it.
ameister14 9y ago
So 3d print modifications to your handgun that alter its shape but not its functionality. Not a difficult workaround.
- adaml_623 9y ago
  Or a piece of paper?
- RUG3Y 9y ago
  simply shroud your firearm in a piece of cloth like the hitmen in gangster movies.
- philipkglass 9y ago
  A lot of common security measures can be defeated with modest technical skill and diligence. Fortunately for the police and the public, the vast majority of armed criminals are not even modestly technically skilled and diligent. They're largely fools with poor impulse control. If you are smart enough to spot weaknesses in security measures and implement workarounds, you probably already have a job with better long term prospects than "bank stickup guy." There are some criminals that get rich from armed robbery, but your typical armed bank robber has a short and none-too-lucrative career: http://www.businessinsider.com/people-are-robbing-banks-in-record-numbers-in-houston-and-most-of-them-are-getting-caught-2012-8 http://www.businessinsider.com/people-are-robbing-banks-in-r... https://www.thestar.com/news/insight/2013/04/26/bank_robbers_why_do_they_even_bother_anymore.html https://www.thestar.com/news/insight/2013/04/26/bank_robbers... https://www.theguardian.com/commentisfree/2014/jul/04/where-did-bank-robbers-go https://www.theguardian.com/commentisfree/2014/jul/04/where-...
  - BoiledCabbage 9y ago
    > A lot of common security measures can be defeated with modest technical skill and diligence. Fortunately for the police and the public, the vast majority of armed criminals are not even modestly technically skilled and diligent. Yes - although almost none of them have the technical expertise to manufacture a gun and yet significant number of criminals own them. Similarly criminals don't need the expertise to own a "gun obfuscating device" to own one.
jjfeore 9y ago
Am I misreading this article. They had 30 test scenes they tested this with, and in those 30 tests it returned 57 false positives? That seems super useless. It sounds like it is basically just throwing alerts for every scene.
- CGamesPlay 9y ago
  The "scenes" are likely hours of video footage, but yes this is certainly missing a denominator. I think the best unit is probably "fraction of video time a handgun is on screen" but that's not told in the article.
obstinate 9y ago
It seems something like this would be great for airport scanners too, although the biggest benefit would be in luggage scanning. I think the challenge would be finding a diverse enough set of true positives to train the model.
colek42 9y ago
I live in the south, it is perfectly legal and socially acceptable to carry a handgun, concealed or open. Something like this could add context to a situation, but sounding an alarm because of a handgun present sounds like a bit much.
turc1656 9y ago
"But what if police officers were equipped with body cameras that could alert them the instant they come into view of a handgun, or some other weapon? Or if bank’s security system could sound an alarm the instant cameras detect a robbery in progress?" Why are we even bothering with spending money trying to create technology like this? How often do banks actually get robbed anymore by gun-wielding maniacs? And how often do officers actually die in the line of duty by getting shot by armed citizens? This is the more frequent case than the bank robbery, and even that is exceedingly rare. Only ~140 officers died on the job in 2016 and of those only 64 died from being shot. Factor in that even in a fair amount of circumstances when they know someone is armed they still end up getting shot and that number of preventable deaths (in theory) gets even lower. What I always find strange is the obsession with "officer safety". Being a cop isn't even one of the top 10 most dangerous jobs in the country, statistically speaking. It's far more dangerous to be in logging, farming, construction, fishing, transportation, etc. And that's comparing the numbers for a very high year like 2016. In 2013 the number of officer deaths was less than half of 2016 - only 31 cops died from being shot - out of ~900,000 officers in the nation. This kind of thing is just a colossal waste of money. "Systems like these still have a ways to go, especially since those false alarms could have deadly consequences for civilians being approached by body camera-wielding cops." That's the biggest issue I have with this. This will become a new reason for them to detain and probably stop and frisk. "The device alerted me to a possible threat" - this will be the excuse and this will lead to reduced rights because the courts will, of course, consider it "reasonable suspicion" based on the technology. There will also be no incentive to increase accuracy since interacting with more citizens makes/catches more criminals, which means numerically they are doing a better job.
- lolsal 9y ago
  > What I always find strange is the obsession with "officer safety". Being a cop isn't even one of the top 10 most dangerous jobs in the country, statistically speaking. It's far more dangerous to be in logging, farming, construction, fishing, transportation, etc. I don't think it has to do with the numbers exactly. If you are a skilled logger and don't make any mistakes, you are likely pretty safe. As a skilled cop, without any mistakes, your life can still be in jeopardy at the drop of a hat. No amount of safety gear can protect a cop from someone pulling a knife or other weapon with intent to harm either the cop or an innocent civilian. I think there is also the aspect that cops are public servants. It's compassion that makes me want to keep these folks alive who are voluntarily taking risks to protect me and my community. I think the numbers are also not totally representative - are all 'cops' lumped in together for the statistic about job safety? Does that include detectives doing mostly office or investigative work? Does it include officers writing parking tickets? Does it blend cops patrolling the suburbs with cops patrolling inner cities? I don't have any sources about my claims here, I don't really know a lot about the dangers of logging or the statistics about the dangers of being a cop. --- What bothers me the most about this is that visual presence of a firearm is immediately associated with criminal behavior or intent.
axelerator 9y ago
I cannot believe no one had pointed out the matching Robocop scene yet: https://www.youtube.com/watch?v=_mqDjcGgE5I https://www.youtube.com/watch?v=_mqDjcGgE5I scnr

29 points by mcspecter 9y ago

> what if police officers were equipped with body cameras that could alert them the instant they come into view of a handgun, or some other weapon? And then > Overall, the optimized model achieved 84% accuracy, with zero false negatives [...] and 57 false positives. So in the what-if scenario above, 16% of cases would falsely escalate a situation dramatically? Seems like a horrible idea. And "really good" seems like a stretch.

mgarfias 9y ago
What happens when it pushes the panic button when a firearm is present, but carried without threat, legally?
mgarfias 9y ago
What happens when it pushes the panic button when a firearm is present, but carried without threat, legally?
codazoda 9y ago
There are also a lot of legal gun owners who have a gun but are not a threat.
- umlaut 9y ago
  That's true in the US, but not in many other places (the UK for example).
- cityhall 9y ago
  They seem to be spotting visible guns, not concealed ones, and the focus is on guns being brandished instead of holstered. This is preliminary work but it wouldn't be useful to have an alarm go off every time another police officer or security guard comes into view with a gun on his hip.
artursapek 9y ago
I guess you'd have to compare the failure rate to that of humans, similar to automated driving.
bdowling 9y ago
The usefulness of a system like this would be where the police have a suspect cornered with their weapons drawn, but the suspect refuses to comply with instructions. If the suspect makes a suspicious move (e.g. as if he is reaching to pull a gun from a back pocket) the police will shoot immediately, since they would not have time to react if they waited to see if he was pulling a gun or not. Right now, the human system provides a 100% false positive rate. A weapon system that incorporates this detector would allow the police to pull the trigger when the suspect makes the suspicious move, but delay the firing of the weapon until the system detects a gun. Even with a 16% false positive rate it would save lives. That said, I don't think police would want it if it might increase the risk to their lives.
- Phlarp 9y ago
  Adding any kind of electronic firing control mechanism to a service pistol is a non-starter for a bunch of reasons: MTBF: Even small guns produce recoil, which over time will damage or degrade any electronics built into the firing mechanism. Security: if a "computer" of whatever complexity is controlling the firing of the gun it presents surface area both for attacks and bugs. Rank and file acceptance: No officer will want to go into the field with a service weapon fitted with "safety restrictions" or "casualty reduction mechanisms" other than the trigger safety.
  - emiliobumachar 9y ago
    I agree with everything you said, except MTBF. The training guns could be separate. Surely the "on-duty" guns are not being fired often enough that this will be an issue at all... right? Right??
    - Phlarp 9y ago
      #1 feeds into #3-- How confident are your rank and file going to be when the "training" guns fail regularly. Also there is something to be said for training with the exact same weapon you plan to use in the field.
burkaman 9y ago
That number should be compared to the rate human officers unnecessarily escalate a situation because they think there's a gun. I don't know that number but it could easily be higher than 16%.
- parsnips 9y ago
  It could easily be lower than 16% too. Much lower.
  - burkaman 9y ago
    Right, I'm saying I don't know the number, but this algorithm shouldn't be judged against a theoretical perfect officer, because that's not reality.
- wayn3 9y ago
  It should really be compared to the number of instances human officer's act as if there could be a gun " just in case ". Like in a traffic stop. If officer's don't have to potentially gun someone down because they made a hectic movement 84% of the time, that's a pretty big win. Imagine 84% of " black guy mysteriously died" actually led to convictions of officers because their ai thing told them "definitely no gun, bro"
  - dragonwriter 9y ago
    > Imagine 84% of " black guy mysteriously died" actually led to convictions of officers because their ai thing told them "definitely no gun, bro" I imagine that, like body cameras, the AI thing will, in those events, conveniently be deactivated, malfunctioning, misplaced, or have it's records lost.
    - wayn3 9y ago
      probably. but you can only pull that stunt so many times.
      - 9y ago
jellicle 9y ago
That sort of accuracy seems like about three orders of magnitude off from what would be useful in the field. Unless you just basically want a drug-sniffing dog, for guns. That is, something to "scienceify" something that would otherwise be problematic (searching this car, or shooting that suspect). "Hey judge, the DOG said it, it must be true!" "Hey judge, the COMPUTER said it, it must be true!"
RUG3Y 9y ago
Beside that, the camera would have to view the subject from the correct angle, so, from the back, most likely position if they're carrying concealed.
tbabb 9y ago
16% false positive is hilariously bad, especially given how rare actual handguns will be. This headline shouldn't be tolerated here.
- jaclaz 9y ago
  On the other hand missing completely it in 3 out of 30 tests (samples with a number of selected "optimizations") is a "terrible" 10% is awfully worse (in an alarm system). >For the experiments, we selected 30 scenes from the previously used videos with the next requirements. Each scene is made up of at least 5 frames, filmed in a fixed scenario, i.e., in the same place, and the pistols are clearly visible to a human viewer. ... The model successfully detects the pistol in 27 scenes with an average time interval AATpI=0.2 seconds, which is good enough for an alarm system. The detector fails to detect pistols only in three scenes. I am probably particularly picky but (from the paper): >This section analyzes the performance of our best detection model on seven public videos, with low qualities, downloaded from youtube. Six of them are pieces of well known films from the 90s, James Bon: The World is Not Enough (video 1), 3 pieces from Pulp Fiction (video 2, 3 and 4), Impossible Mission: Rogue Nation (video 5) and Mister Bin (video 6). The seventh video is a long pistol threatening video (video 7). ... if you cannot spell Bond or Mister Bean, and you classify Mission Impossible: Rogue Nation as a 90's movie (calling it Impossible Mission) you don't look like having been particularly accurate when writing the article, if the same accurateness is in the actual numbers, there is a problem. Anyway the "final" sample (30 scenes) is so small that it makes very little sense to derive anything from it.

ameister14 9y ago

So 3d print modifications to your handgun that alter its shape but not its functionality. Not a difficult workaround.

adaml_623 9y ago
Or a piece of paper?
RUG3Y 9y ago
simply shroud your firearm in a piece of cloth like the hitmen in gangster movies.
philipkglass 9y ago
A lot of common security measures can be defeated with modest technical skill and diligence. Fortunately for the police and the public, the vast majority of armed criminals are not even modestly technically skilled and diligent. They're largely fools with poor impulse control. If you are smart enough to spot weaknesses in security measures and implement workarounds, you probably already have a job with better long term prospects than "bank stickup guy." There are some criminals that get rich from armed robbery, but your typical armed bank robber has a short and none-too-lucrative career: http://www.businessinsider.com/people-are-robbing-banks-in-record-numbers-in-houston-and-most-of-them-are-getting-caught-2012-8 http://www.businessinsider.com/people-are-robbing-banks-in-r... https://www.thestar.com/news/insight/2013/04/26/bank_robbers_why_do_they_even_bother_anymore.html https://www.thestar.com/news/insight/2013/04/26/bank_robbers... https://www.theguardian.com/commentisfree/2014/jul/04/where-did-bank-robbers-go https://www.theguardian.com/commentisfree/2014/jul/04/where-...
- BoiledCabbage 9y ago
  > A lot of common security measures can be defeated with modest technical skill and diligence. Fortunately for the police and the public, the vast majority of armed criminals are not even modestly technically skilled and diligent. Yes - although almost none of them have the technical expertise to manufacture a gun and yet significant number of criminals own them. Similarly criminals don't need the expertise to own a "gun obfuscating device" to own one.

jjfeore 9y ago

Am I misreading this article. They had 30 test scenes they tested this with, and in those 30 tests it returned 57 false positives? That seems super useless. It sounds like it is basically just throwing alerts for every scene.

CGamesPlay 9y ago
The "scenes" are likely hours of video footage, but yes this is certainly missing a denominator. I think the best unit is probably "fraction of video time a handgun is on screen" but that's not told in the article.

obstinate 9y ago

It seems something like this would be great for airport scanners too, although the biggest benefit would be in luggage scanning. I think the challenge would be finding a diverse enough set of true positives to train the model.

colek42 9y ago

I live in the south, it is perfectly legal and socially acceptable to carry a handgun, concealed or open. Something like this could add context to a situation, but sounding an alarm because of a handgun present sounds like a bit much.

turc1656 9y ago

"But what if police officers were equipped with body cameras that could alert them the instant they come into view of a handgun, or some other weapon? Or if bank’s security system could sound an alarm the instant cameras detect a robbery in progress?" Why are we even bothering with spending money trying to create technology like this? How often do banks actually get robbed anymore by gun-wielding maniacs? And how often do officers actually die in the line of duty by getting shot by armed citizens? This is the more frequent case than the bank robbery, and even that is exceedingly rare. Only ~140 officers died on the job in 2016 and of those only 64 died from being shot. Factor in that even in a fair amount of circumstances when they know someone is armed they still end up getting shot and that number of preventable deaths (in theory) gets even lower. What I always find strange is the obsession with "officer safety". Being a cop isn't even one of the top 10 most dangerous jobs in the country, statistically speaking. It's far more dangerous to be in logging, farming, construction, fishing, transportation, etc. And that's comparing the numbers for a very high year like 2016. In 2013 the number of officer deaths was less than half of 2016 - only 31 cops died from being shot - out of ~900,000 officers in the nation. This kind of thing is just a colossal waste of money. "Systems like these still have a ways to go, especially since those false alarms could have deadly consequences for civilians being approached by body camera-wielding cops." That's the biggest issue I have with this. This will become a new reason for them to detain and probably stop and frisk. "The device alerted me to a possible threat" - this will be the excuse and this will lead to reduced rights because the courts will, of course, consider it "reasonable suspicion" based on the technology. There will also be no incentive to increase accuracy since interacting with more citizens makes/catches more criminals, which means numerically they are doing a better job.

lolsal 9y ago
> What I always find strange is the obsession with "officer safety". Being a cop isn't even one of the top 10 most dangerous jobs in the country, statistically speaking. It's far more dangerous to be in logging, farming, construction, fishing, transportation, etc. I don't think it has to do with the numbers exactly. If you are a skilled logger and don't make any mistakes, you are likely pretty safe. As a skilled cop, without any mistakes, your life can still be in jeopardy at the drop of a hat. No amount of safety gear can protect a cop from someone pulling a knife or other weapon with intent to harm either the cop or an innocent civilian. I think there is also the aspect that cops are public servants. It's compassion that makes me want to keep these folks alive who are voluntarily taking risks to protect me and my community. I think the numbers are also not totally representative - are all 'cops' lumped in together for the statistic about job safety? Does that include detectives doing mostly office or investigative work? Does it include officers writing parking tickets? Does it blend cops patrolling the suburbs with cops patrolling inner cities? I don't have any sources about my claims here, I don't really know a lot about the dangers of logging or the statistics about the dangers of being a cop. --- What bothers me the most about this is that visual presence of a firearm is immediately associated with criminal behavior or intent.

axelerator 9y ago

I cannot believe no one had pointed out the matching Robocop scene yet: https://www.youtube.com/watch?v=_mqDjcGgE5I https://www.youtube.com/watch?v=_mqDjcGgE5I scnr