5 ms·

Multiple concert venues in my city use these, so I interact with them all the time. They have replaced standard metal detectors, bag searches, and manual patdow

Multiple concert venues in my city use these, so I interact with them all the time. They have replaced standard metal detectors, bag searches, and manual patdowns w/ hands and/or metal-detecting wands. Security checkpoints are the primary point of delay for getting into venues, and places that have rolled these out process people through about 95% faster. It's a huge difference. If it does trigger, you just get the manual patdown you would have gotten anyway, so the false positive cases aren't any lost time. The article and settlement seem to only mention the false positive rate, which is a bad thing to focus on. Every true positive is a much faster experience. Only subjecting 110 out of 3000 people to a longer search is a big improvement. Given the negative outcomes of a gun slipping through and the lack of a cost of a false positive, we probably want it to be tuned to be more false positive prone anyway. We don't need these to detect guns THAT well, we just need them to weed out people who definitely don't have them. I do have concerns about what its false negative rate is relative to the standard practice it replaces. I do not really trust whatever psuedo-AI they're bolting to their metal detectors; it's probably easier to get a gun through. That said, the false negative rate probably isn't good already. TSA isn't great on their false positive rate, does more intense screening, and isn't being staffed by hungover 20-somethings. So maybe the false negative rate didn't actually increase by much?

ggreer 2y ago
An acceptable false negative rate really depends on the consequences of getting caught. If someone's nefarious plan depends on smuggling a gun in, they want to be confident they won't be arrested or shot at the entrance. Even failing to detect 20% of firearms means there's an 80% chance they'll be caught before they can do whatever it is they plan on doing. This is also why it's important to have armed guards alongside the scanners. Scanners aren't very useful if the only armed person is the bad guy. If the consequences of getting caught are negligible (as is the case for anyone trying to bring a box cutter through airport security), then the attacker can try as many times as they want without issue. Even if the false negative rate is low, they only have to get lucky once. Annoyingly, I can't find any info about false positive/negative rates for various scanners. There doesn't seem to be the equivalent of Consumer Reports or Underwriters Labs for scanners. My guess is that the numbers must be pretty bad if companies aren't willing to go through public 3rd party testing.
- mrandish 2y ago
  The TSA's own security tests clearly show a significant percentage of guns, knives and explosives regularly get through. This is further confirmed by the number of travelers who, after arriving, discover the handgun they accidentally left in some pouch in their suitcase that was never detected. > There doesn't seem to be the equivalent of Consumer Reports or Underwriters Labs for scanners. My guess is that the numbers must be pretty bad if companies aren't willing to go through public 3rd party testing. Of course they are but the main reason there's no publicly available objective testing isn't only that sellers don't want it. In reality, no stakeholder in the security market wants it. The vast majority of high-volume public security like airports, concerts and sporting events is largely unnecessary and mostly ineffective but our current political/media environment requires appearing to "do something" to "make it safe". The Vice-President of "Make it (Seem) Safe" knows that their shareholders, politicians and the public aren't willing to pay more or be even more inconvenienced than they already are for 800% better "Make it (Seem) Safe"-ness. Metaphorically speaking, the tiger repellent is working just fine, thank you. Those truly worried about tiger attacks feel safer and those being well-paid for preventing tiger attacks can claim virtually 100% effectiveness. So, if you start the world's best Tiger Repellent Testing Laboratory, you'll find a shocking lack of interest in buying your test data from both sellers and buyers in this brisk, profitable and growing market. Much like the lack of interest in objective testing data for lie detectors, astrology readings and placebo pills. The smaller minority of customers actually willing to pay more for improved detection (like Tel Aviv airport), do their own in-context performance testing anyway. In fact, a good proxy for doing your own effectiveness testing is available for free. Just look at what those under constant active threat with real consequences actually pay for and do.
  - yonaguska 2y ago
    I unknowingly transported ammo both to and from Mexico. I used an old backpack that I had previously used as a range bag from years ago. I ended up finding several 223 rounds in Mexico, then when I got back, even more 22lr.
    - bwilliams18 2y ago
mrandish 2y ago
> Security checkpoints are the primary point of delay for getting into venues, and places that have rolled these out process people through about 95% faster. It's a huge difference. I assume expediting peak crowd throughput at low labor cost is the primary, if not entire, value of the device. I hate that it's being marketed dishonestly but I also assume most concert venue buyers know (or suspect) it probably doesn't work all that well in practice. However, in a concert context accurate detection isn't their main priority. They need to get more bodies per minute into the venue at lower cost while appearing to conduct security checks sufficiently 'real' enough to act as a deterrent to get those who care about getting 'caught' to leave their knife or concealed carry handgun (or whatever) in the car. The only hard and fast requirement is meeting the contractual security requirements of the venue and promoter's insurance carriers - because no insurance = no concert. It's a bonus if the 'security' also looks plausible enough to reassure the small fraction of perpetually fearful people statistically challenged enough to actually worry about terrorists or an active shooter killing them while at a Taylor Swift concert (as opposed to the infinitely more likely chance of dying in a car crash on the way to the concert). In a perfect world, everyone would be rational and numerate enough that we wouldn't need to maintain the pretense of 'security theater' in contexts where actual security isn't necessary. But in the imperfect world we live in, I prefer having concert (and airport) security be as minimally disruptive and inexpensive as possible regardless of effectiveness (since it's unnecessary and mostly ineffective in those contexts anyway). I just wish companies would sell these products as 'security placebos' instead of lying about it because fraud is wrong.
- mlyle 2y ago
  I don't know anything about the devices that are being sold, but it doesn't seem impossible to me that with better signal processing from a metal detector, you could reduce false positives a bit while maintaining or slightly improving the false negative rate.
  - mrandish 2y ago
    Sure, I agree that's an interesting and likely solvable technical problem. However, the vast majority of the addressable market in today's over-secured society don't really need improved detection. Concert venues, sports arenas and similar customers buy massive volume and they are much more concerned with faster throughput enabled by shorter cycle time and minimal false positives. Of course the head of security at Madison Square Garden can never publicly admit they don't care about better detection enough to pay more for it, but I'm confident the sales managers at these security vendors understand exactly what their largest market segments really care about. Customers like Tel Aviv International Airport, who actually care to some meaningful extent about improved detection, are a small minority segment of the overall market. Creating new technical measures able to demonstrate improved performance in rigorous objective tests on the metrics these customers care about (some sweeter spot on the matrix of false pos, false neg, true pos, true neg, net throughput, cost) would be valuable but only to that small segment.
LordDragonfang 2y ago
>TSA isn't great on their false positive rate, does more intense screening, and isn't being staffed by hungover 20-somethings. So maybe the false negative rate didn't actually increase by much? TSA is abysmal on the false negative rate for things that actually matter. The FNR for actual weapons and explosives is somewhere between 80 and 95%[1]. It's because they waste all of their attention looking for nail clippers and water bottles. Even an FNR of 50% would be a massive improvement. [1] https://abcnews.go.com/US/tsa-fails-tests-latest-undercover-operation-us-airports/story?id=51022188 https://abcnews.go.com/US/tsa-fails-tests-latest-undercover-...
dawnerd 2y ago
Theme parks are switching to these too and it's oh so much nicer to just walk through and if you're unlucky having your bag searched.
pxmpxm 2y ago
I guess on a more general level, I'm confused why a metal detector with bit some machine learning on top wouldn't make a better widget? You'd think that different shape metal object produce different magnetic flux, and there's probably more dimensions to that than just size?
- imtringued 2y ago
  If it's just a gate that you are walking through then it's all or nothing. Basically just a metal detector. If you have a conveyor belt system and a CT scanner like setup then yes, you could build a better metal detector.

ggreer 2y ago

An acceptable false negative rate really depends on the consequences of getting caught. If someone's nefarious plan depends on smuggling a gun in, they want to be confident they won't be arrested or shot at the entrance. Even failing to detect 20% of firearms means there's an 80% chance they'll be caught before they can do whatever it is they plan on doing. This is also why it's important to have armed guards alongside the scanners. Scanners aren't very useful if the only armed person is the bad guy. If the consequences of getting caught are negligible (as is the case for anyone trying to bring a box cutter through airport security), then the attacker can try as many times as they want without issue. Even if the false negative rate is low, they only have to get lucky once. Annoyingly, I can't find any info about false positive/negative rates for various scanners. There doesn't seem to be the equivalent of Consumer Reports or Underwriters Labs for scanners. My guess is that the numbers must be pretty bad if companies aren't willing to go through public 3rd party testing.

mrandish 2y ago
The TSA's own security tests clearly show a significant percentage of guns, knives and explosives regularly get through. This is further confirmed by the number of travelers who, after arriving, discover the handgun they accidentally left in some pouch in their suitcase that was never detected. > There doesn't seem to be the equivalent of Consumer Reports or Underwriters Labs for scanners. My guess is that the numbers must be pretty bad if companies aren't willing to go through public 3rd party testing. Of course they are but the main reason there's no publicly available objective testing isn't only that sellers don't want it. In reality, no stakeholder in the security market wants it. The vast majority of high-volume public security like airports, concerts and sporting events is largely unnecessary and mostly ineffective but our current political/media environment requires appearing to "do something" to "make it safe". The Vice-President of "Make it (Seem) Safe" knows that their shareholders, politicians and the public aren't willing to pay more or be even more inconvenienced than they already are for 800% better "Make it (Seem) Safe"-ness. Metaphorically speaking, the tiger repellent is working just fine, thank you. Those truly worried about tiger attacks feel safer and those being well-paid for preventing tiger attacks can claim virtually 100% effectiveness. So, if you start the world's best Tiger Repellent Testing Laboratory, you'll find a shocking lack of interest in buying your test data from both sellers and buyers in this brisk, profitable and growing market. Much like the lack of interest in objective testing data for lie detectors, astrology readings and placebo pills. The smaller minority of customers actually willing to pay more for improved detection (like Tel Aviv airport), do their own in-context performance testing anyway. In fact, a good proxy for doing your own effectiveness testing is available for free. Just look at what those under constant active threat with real consequences actually pay for and do.
- yonaguska 2y ago
  I unknowingly transported ammo both to and from Mexico. I used an old backpack that I had previously used as a range bag from years ago. I ended up finding several 223 rounds in Mexico, then when I got back, even more 22lr.
  - bwilliams18 2y ago

mrandish 2y ago

> Security checkpoints are the primary point of delay for getting into venues, and places that have rolled these out process people through about 95% faster. It's a huge difference. I assume expediting peak crowd throughput at low labor cost is the primary, if not entire, value of the device. I hate that it's being marketed dishonestly but I also assume most concert venue buyers know (or suspect) it probably doesn't work all that well in practice. However, in a concert context accurate detection isn't their main priority. They need to get more bodies per minute into the venue at lower cost while appearing to conduct security checks sufficiently 'real' enough to act as a deterrent to get those who care about getting 'caught' to leave their knife or concealed carry handgun (or whatever) in the car. The only hard and fast requirement is meeting the contractual security requirements of the venue and promoter's insurance carriers - because no insurance = no concert. It's a bonus if the 'security' also looks plausible enough to reassure the small fraction of perpetually fearful people statistically challenged enough to actually worry about terrorists or an active shooter killing them while at a Taylor Swift concert (as opposed to the infinitely more likely chance of dying in a car crash on the way to the concert). In a perfect world, everyone would be rational and numerate enough that we wouldn't need to maintain the pretense of 'security theater' in contexts where actual security isn't necessary. But in the imperfect world we live in, I prefer having concert (and airport) security be as minimally disruptive and inexpensive as possible regardless of effectiveness (since it's unnecessary and mostly ineffective in those contexts anyway). I just wish companies would sell these products as 'security placebos' instead of lying about it because fraud is wrong.

mlyle 2y ago
I don't know anything about the devices that are being sold, but it doesn't seem impossible to me that with better signal processing from a metal detector, you could reduce false positives a bit while maintaining or slightly improving the false negative rate.
- mrandish 2y ago
  Sure, I agree that's an interesting and likely solvable technical problem. However, the vast majority of the addressable market in today's over-secured society don't really need improved detection. Concert venues, sports arenas and similar customers buy massive volume and they are much more concerned with faster throughput enabled by shorter cycle time and minimal false positives. Of course the head of security at Madison Square Garden can never publicly admit they don't care about better detection enough to pay more for it, but I'm confident the sales managers at these security vendors understand exactly what their largest market segments really care about. Customers like Tel Aviv International Airport, who actually care to some meaningful extent about improved detection, are a small minority segment of the overall market. Creating new technical measures able to demonstrate improved performance in rigorous objective tests on the metrics these customers care about (some sweeter spot on the matrix of false pos, false neg, true pos, true neg, net throughput, cost) would be valuable but only to that small segment.

LordDragonfang 2y ago

>TSA isn't great on their false positive rate, does more intense screening, and isn't being staffed by hungover 20-somethings. So maybe the false negative rate didn't actually increase by much? TSA is abysmal on the false negative rate for things that actually matter. The FNR for actual weapons and explosives is somewhere between 80 and 95%[1]. It's because they waste all of their attention looking for nail clippers and water bottles. Even an FNR of 50% would be a massive improvement. [1] https://abcnews.go.com/US/tsa-fails-tests-latest-undercover-operation-us-airports/story?id=51022188 https://abcnews.go.com/US/tsa-fails-tests-latest-undercover-...

dawnerd 2y ago

Theme parks are switching to these too and it's oh so much nicer to just walk through and if you're unlucky having your bag searched.

pxmpxm 2y ago

I guess on a more general level, I'm confused why a metal detector with bit some machine learning on top wouldn't make a better widget? You'd think that different shape metal object produce different magnetic flux, and there's probably more dimensions to that than just size?

imtringued 2y ago
If it's just a gate that you are walking through then it's all or nothing. Basically just a metal detector. If you have a conveyor belt system and a CT scanner like setup then yes, you could build a better metal detector.