8 ms·

Facial Recognition Failed in Boston Bombing

98 points by jhonovich 14y ago

aashaykumar92 14y ago
When the FBI released the photos of the two suspects, I kept wondering, "Why do they need the publics help when I'm sure there is facial recognition software to help match the suspects' faces?" Well I guess this article somewhat answers this, but I am still in disbelief that our government does not have better software. For how long it took to discover their names, the two suspects had TOO much time to escape. They had about 48 hours to book flights out of the country, flee to another part of the country, do more damage, etc. The lack of facial recognition software could have ultimately led to this case being unnecessarily longer. This is an issue that CAN and NEEDS to be fixed quickly--in fact, I hope the government is working on a more sophisticated facial recognition software right now. Ideally, the facial recognition software should be able to crawl through the web to search for faces. Now that I think about it, a good hacker could build a search engine that allows people to post photos and the program will run an automated search across the web (Social networks, Google, etc.) to see which pictures match it. Hell, Facebook even has facial recognition to some extent. A more sophisticated one cannot be that difficult for the government to build and put to use.
- mjn 14y ago
  There is a lot of money being put into facial recognition software by law enforcement and the military (including DARPA), so it's certainly the case that it's being worked on. But it's a hard problem, especially to do robustly at a large scale. What they do have is almost certainly the state of the art— in fact much of the current commercial state-of-the-art is itself based on stuff developed in older DARPA projects.
- sp332 14y ago
  Facebook only has to distinguish among friends of the uploader. And it's not perfect. This man was wearing sunglasses and a baseball cap. He would not have been identifiable out of the tens of thousands of people at the event, matched across the (probably) millions of people in the database.
  - sliverstorm 14y ago
    Plus Facebook gets high resolution pictures of people's faces, and asks for confirmation. It doesn't get it right every time.
- sy_r0y 14y ago
  I get what you mean when you say these guys had way too much time to make some kind of an escape and I agree then that there is an absolute need for a much better software. Having said that, I do not think it would be wise for government to build and use an "image crawler" to search and store images of citizens...it would provoke widespread criticism concerning privacy of millions of citizens. On top of that in this particular instance, the police commissioner points out that "both Tsarnaevs’ images exist in official databases"..the software failed to make a match.
- sehugg 14y ago
  What's wrong with needing the public's help? The public did a great job in providing critical evidence. Maybe instead of more intrusive surveillance we find more ways for the public to participate. Maybe we uphold the public's right to take pictures and video in "sensitive areas". Etc.
bsenftner 14y ago
My big questions is why were the suspects faces in the database? Why would a 19 year old, never done anything face be in the facial recognition database?
- dr_doom 14y ago
  The older brother had been questioned by the FBI before. http://www.cbsnews.com/8301-201_162-57580534/fbi-interviewed-dead-boston-bombing-suspect-years-ago/ http://www.cbsnews.com/8301-201_162-57580534/fbi-interviewed...
- DannyBee 14y ago
  Best random guess: Something related to him being an alien and becoming a naturalized citizen
  - DannyBee 14y ago
    Actually, figured it out: Dzhokhar had a Massachusetts driver’s license, and those pictures go into a facial recognition database. Source: http://www.washingtonpost.com/world/national-security/inside-the-investigation-of-the-boston-marathon-bombing/2013/04/20/19d8c322-a8ff-11e2-b029-8fb7e977ef71_print.html http://www.washingtonpost.com/world/national-security/inside...
- sp332 14y ago
  He's 19, so he likely has a driver's license or other state-issued ID photo on file.
- ics 14y ago
  I'm going to venture to guess that either a) the officer was talking out of his ass to make a quick point or b) someone, somewhere massively scraped every social network they could think of for anyone with ties to the area or the event. That being said, I don't really like the sound of it either, regardless of how true it is. --pre-submit thought: perhaps it was the older brother? Our background is rather limited but it would not be surprising if he had had prior run-ins with the law. Even a minor offense will get your mugshot in the file.
mrkmcknz 14y ago
Does anyone have detailed knowledge of startups in this space? I know of Lambda labs but I personally haven't use their solution post Face.com.
- brk 14y ago
  There are a few, though less than you might think. Lots of general progress has been made in terms of facial detection algorithms, but there are almost no "general" security camera deployments that are setup to capture sufficient detail to get a good face match. There are not enough events like this in open places that drive requirements for facial recognition use. Limited cases (airports, casinos, etc.) more frequently have cameras setup to get good straight-on high-res facial shots, but almost no typical outdoor area is covered that way.
jhonovich 14y ago
I am the author of the referenced article. The problem is matching surveillance camera images (like from Lord & Taylor's) is impossible because of the steep down tilt angle of the cameras plus their low resolution. Smartphone images could be a little better as the angle is more head on and the resolution is higher. However, even those images tended to be in a wide field of view which means low pixel density, hurting matching performance. Last, but not least, the caps and sunglasses hurt recognition as well.
- xfs 14y ago
  Matching surveillance camera images is always possible no matter how bad the image quality is. The only matter is how many false positives show up. Facial recognition algorithms always give their best matching candidates, with thresholds to skip those not quite matching ones. Your article makes it sound like no such results were produced at all, which is unlikely how recognition is supposed to work.
  - onli 14y ago
    Not impossible though. Given that such software is used to steer the future efforts of the investigators, it is possible that the software does say "no match found" if the probability of a real match is too low.
  - jhonovich 14y ago
    First of all, almost any facial surveillance system would simply reject images like those from Lord & Taylor's. The pixel density is simply too low to even attempt plus the downtilt would also likely trigger a rejection. That's how facial recognition does work in practice. Even if the system would allow it, not only would false positives be enormous, you'd probably miss the match as well (false negative).
- akennberg 14y ago
  Interesting to see articles promoting adding more surveillance cameras for all their flaws (bad angle, resolution trails behind phone cameras, etc). I would focus on acquiring publicly available photo data in real time at dense events and scan them. I.e. crowdsourced surveillance. I'm sure someone is working on that.
marknutter 14y ago
How hard would it be to throw off facial recognition by simply wearing very high quality makeup and prosthetics to alter your facial appearance (ala hollywood movies)? If I were the boston bombers, I would have first learned how to make myself look completely different. It's unbelievable to me that these guys didn't realize they'd be captured on tons of cameras that day.
- nwh 14y ago
  If you wore Mission Impossible style masks, there's no reason why the facial recognition would be able to pick you up from underneath. You might alert the suspicion of people around you more though, as they don't ever seem to be perfect.
  - runarb 14y ago
    What if you wore something like this: http://www.spfxmasks.com/ourmasks.php http://www.spfxmasks.com/ourmasks.php ? One should be able to blend in, and make video surveillance pretty useless... Edit: Did work for this robbers: http://www.dailymail.co.uk/news/article-2192115/Robbers-disguised-white-cops-caught--send-polite-thank-letter-company-unbelievable-latex-masks.html http://www.dailymail.co.uk/news/article-2192115/Robbers-disg...
- Plimsoll 14y ago
  This brings to mind this bank robbery news from few years aback: http://gizmodo.com/5711252/white-guy-becomes-black-bank-robber-with-super-realistic-hollywood-mask http://gizmodo.com/5711252/white-guy-becomes-black-bank-robb... Don't know how easy it is to get those masks though.
- speednoise 14y ago
  Easier than you may think: http://cvdazzle.com/ http://cvdazzle.com/
- gcr 14y ago
  CV-Dazzle attempts to throw off face detectors by doing just that: http://cvdazzle.com/ http://cvdazzle.com/ But then again, the police will label you as "that guy with weird bits of plastic on his face", which won't help you hide at all.
jere 14y ago
I watched this lecture recently http://youtu.be/-IUe3ce_29I http://youtu.be/-IUe3ce_29I which talked about how hard it is to do image recognition at low resolution. One solution is a "fovea" system that scans an area for interesting items and zooms in on them with a high resolution camera automatically to get better recognition. I wonder if this would help in similar instances. A system that uses facial detection to isolate faces and then zoom in on them very quickly seems feasible. I'm assuming it would be overwhelmed in a scenario like a marathon of course, but perhaps it would help in other situations.
eksith 14y ago
Bottom line is that it's nearly impossible to exactly match facial features to anyone beyond a set distance when there's too much ambiguity between distances and rubbish quality CCTV video. Matching gait may be a more effective means of identification at a distance (and, in this and many other cases, at odd angles). http://www.am.sanken.osaka-u.ac.jp/~sagawa/pdf/accv06.pdf http://www.am.sanken.osaka-u.ac.jp/~sagawa/pdf/accv06.pdf (PDF)
jpalomaki 14y ago
I can understand that recording live footage with very high quality is a problem, but how about capturing quality stills to complement the video with lower quality.
- emiliobumachar 14y ago
  Someone with knowledge of cameras please either correct or endorse this, but I think the largest price/quality tradeoff in any camera happens at the lens choice, and given a cheap lens, the stills can't be much better than the video.
  - securingsincity 14y ago
    certainly the lens and sensor are the biggest trade offs with price these would have the biggest impact with image clarity and the amount of light taken in to take the image.
  - tripzilch 14y ago
    I think that is right (I can't claim much specific knowledge on camera lenses though), it used to be especially bad with the whole "megapixel wars" a few years back. Pixel resolution increased, but image clarity didn't, and you would get much better images on old low-megapixel cameras with relatively high-end lenses than with a cheap but new camera of approx 4-8x the resolution in pixels. However, nowadays I am quite amazed by the picture quality of the average smartphone camera. So maybe something changed? Still, we're talking about better lenses/quality than smartphone photos. They may be sharp, but that comes from the advantage of being up close and coming in huge numbers. And in addition to the lens, there's a lot of other expensive parts in a security system, that may need to be replaced or reconfigured because it is all integrated with the current low-res cameras. The bigger such a system is, the more expensive it becomes to change it all up when better tech has come around, even if the new parts are relatively cheap by themselves.
  - brk 14y ago
    Not exactly. A lot of it has to do with the fact that most security cameras are not setup for capture of facial details (or license plates for that matter). Most security cameras are setup for general overview shots, with around 10-20ppf (pixels per foot), whereas good detail starts at around 80ppf (meaning that you'd need 4x as many cameras in most areas to start to get something approaching an image quality that you could use for reliable facial recognition. On top of that there is the storage factor. A "common" camera in a new deployment is a 1080p (2.1 Megapixel) camera. If this is set to record good quality images 24/7, it consumes about 60GB of storage per day. Various tricks like "record on motion", etc. can reduce this to maybe 10GB/day, at the risk of potentially missing key evidence or having gaps in the recording. Lenses are important, but it's not that hard to find a lens with sufficient resolution for these cameras. A good lens rated at 2+ megapixels is around $70-$120 (depending on various factors), and a good outdoor camera (including lens) can be had for $500-$1200. Big price ranges, yes, but not to the point that the lenses are the limiting factor budget-wise.
mikecane 14y ago
How to camouflage yourself from facial recognition technology http://venturebeat.com/2010/07/02/facial-recognition-camouflage/ http://venturebeat.com/2010/07/02/facial-recognition-camoufl...
- anigbrowl 14y ago
  That's not conspicuous at all. And like zebras, there's no benefit unless everyone else is doing it.
  - okamiueru 14y ago
    Dazzle camouflage isn't meant to hide you in your surroundings, but rather make it more difficult to assess the geometry -- how large you are, where you are heading, etc. It's more for confusing the lion running after the zebra, than an attempt hide from the lion.
ck2 14y ago
Not just facial recognition, the "terror watch list" failed (after a warning from Russia about the older brother) lack of background checks on the AR15s they owned failed, the secured perimeter failed (the teenager was outside of it) the city lockdown failed (the guy smoking the cigarette outside when he saw the blood was technically violating the lockdown). In the end it was mostly dumb luck, dumb criminals, and the fact the poor victim who lost their legs managed to survive. In fact that was the success, that more wounded people didn't die thanks to fantastic work by the hospital and doctors. Instead of patting law enforcement on their backs, how about thanking the doctors, where is their applause?
- afarrell 14y ago
  Actually, the lockdown was voluntary, though with strong community pressure to comply. But yes, multiple subsystems failed in the police response. If you listened to police scanners, you also heard that they had some trouble identifying streets correctly, or keeping their mics off when not talking. But police response protocols are one human system that definitely needs to be able to work in the face of multiple subsystem failure. I say judge the police on how the response worked as a whole. I have to say though, thank god that Dunkin' Doughnuts' Tactical Ration Deployment Vehicle ran smoothly.
  - ck2 14y ago
    Talking about the scanners and technology (and failures) didn't "homeland security" give millions to metro police departments around the country to get their communication encrypted? I vaguely remember reading about that a couple years ago. I wonder where that money went if that is accurate. https://www.google.com/search?q=%22federal+grant%22+%22encrypted+radios%22 https://www.google.com/search?q=%22federal+grant%22+%22encry...
    - datr 14y ago
      I've wanted to ask about this since seeing the Boston police channel available on the internet. In the UK all the channels are encrypted making it virtually impossible for a civilian to listen in. Is it simply financial reasons that the USA doesn't do this?
astar 14y ago
Facial recognition is still treated like some magical technology, like the "zoom in, enhance" feature much mocked about on TV crime shows. I bet Facebook's technology is ahead of law enforcement's in terms of data, algorithm, and refinement, and yet it still stumbles on pretty obvious faces (if you have the default auto-tag feature left on)
- brk 14y ago
  Correct, and you're talking about cases where you might have literally 500,000+ pixels on a face (close-up shot from an 8MP camera). The average cctv shot will have around 200-1000 pixels on the entire face, if you're lucky.
- gcr 14y ago
  Facebook only has to distinguish between friends of the uploader. US law enforcement has to distinguish between 300 million civilians. Of the two, I would expect Facebook to be more successful at identifying peolpe.
- anigbrowl 14y ago
  "zoom in, enhance" feature much mocked about on TV crime shows Actually, those jokes refer to a scene in Blade Runner: https://www.youtube.com/watch?v=qHepKd38pr0 https://www.youtube.com/watch?v=qHepKd38pr0
  - socillion 14y ago
    NTSF:SD:SUV Parody of the 'enhance' trope in cop drama setting: https://www.youtube.com/watch?v=xbCWYm7B_B4&t=58s https://www.youtube.com/watch?v=xbCWYm7B_B4&t=58s
WestCoastJustin 14y ago
Why did it fail? That, to me, is the most interesting unanswered question.
- new_test 14y ago
  Because facial recognition is a very difficult problem.
  - sliverstorm 14y ago
    Surely all you need to do is start an agile startup and throw some Ruby on Rails at the problem?
gcr 14y ago
The problem is that any non-perfect automated system will wrongly convict so many civilians as being terrorists that nobody will trust it anymore. Let's say that you want to catch terrorists creeping around in your airport. To do this, you have a magic camera that, when it takes a picture of a civilian, compares it to a picture of a known terrorist. If they're in fact the same person, it will flag security over. Let's say that it is indeed a magic camera and makes the correct decision 99.99% of the time, which is far better than the current state of the art[1]. Do you have any idea how much DHS or DoD would pay for such a magic camera? But it won't be enough. Consider a watch list of 100 terrorists that we're looking for in our airport. If our camera compares each civilian's picture to all 100 terrorists in succession, and if we model each comparison as a Bernoulli trial, then each civilian has a 0.9999^100 = 99.004% chance of getting through security unfazed, which means we flag innocent people as terrorists about 1% of the time. This doesn't work in an airport setting. Assuming 5,000 people pass through your small airport per hour and you have a 14-hour day, that means your security people have to go through 0.01 * 5000 * 14 = 700 false alarms each day. That's about 1.2 false alarms per minute. Your security people will have to go through all their checks each time this happens. Before long, they're going to stop trusting your magic camera. As you scale your watch list up, it gets worse. How well would it work for 1,000 people on the watch list? 10,000? Note that the current US watch list has 50,000 people on it. Even if it were a perfect magic camera, it coudln't catch terrorists on their maiden crime who aren't on the watch list at all. (Most of this post was adapted from: http://vast.uccs.edu/~tboult/tmp/IJCB11-tutorial-boult.ppt http://vast.uccs.edu/~tboult/tmp/IJCB11-tutorial-boult.ppt , which is a really interesting presentation if you're interested in these sorts of things.) 1: http://vis-www.cs.umass.edu/lfw/results.html http://vis-www.cs.umass.edu/lfw/results.html
- stretchwithme 14y ago
  Do you know how unreliably people recognize strangers? Many people are already wrongly convicted on such evidence. Even perfect automated facial recognition won't convict people. It will merely point out people that should be looked at. If a jury looking at the image and the guy's face think the system is wrong, they can disregard it. As far as prevention goes, what we need are systems that can detect deception reliably, which will help if we ever decide to ask people meaningful questions. Not to convict, but to focus attention on a possible threat.

98 points by jhonovich 14y ago

When the FBI released the photos of the two suspects, I kept wondering, "Why do they need the publics help when I'm sure there is facial recognition software to help match the suspects' faces?" Well I guess this article somewhat answers this, but I am still in disbelief that our government does not have better software. For how long it took to discover their names, the two suspects had TOO much time to escape. They had about 48 hours to book flights out of the country, flee to another part of the country, do more damage, etc. The lack of facial recognition software could have ultimately led to this case being unnecessarily longer. This is an issue that CAN and NEEDS to be fixed quickly--in fact, I hope the government is working on a more sophisticated facial recognition software right now. Ideally, the facial recognition software should be able to crawl through the web to search for faces. Now that I think about it, a good hacker could build a search engine that allows people to post photos and the program will run an automated search across the web (Social networks, Google, etc.) to see which pictures match it. Hell, Facebook even has facial recognition to some extent. A more sophisticated one cannot be that difficult for the government to build and put to use.

mjn 14y ago
There is a lot of money being put into facial recognition software by law enforcement and the military (including DARPA), so it's certainly the case that it's being worked on. But it's a hard problem, especially to do robustly at a large scale. What they do have is almost certainly the state of the art— in fact much of the current commercial state-of-the-art is itself based on stuff developed in older DARPA projects.
sp332 14y ago
Facebook only has to distinguish among friends of the uploader. And it's not perfect. This man was wearing sunglasses and a baseball cap. He would not have been identifiable out of the tens of thousands of people at the event, matched across the (probably) millions of people in the database.
- sliverstorm 14y ago
  Plus Facebook gets high resolution pictures of people's faces, and asks for confirmation. It doesn't get it right every time.
sy_r0y 14y ago
I get what you mean when you say these guys had way too much time to make some kind of an escape and I agree then that there is an absolute need for a much better software. Having said that, I do not think it would be wise for government to build and use an "image crawler" to search and store images of citizens...it would provoke widespread criticism concerning privacy of millions of citizens. On top of that in this particular instance, the police commissioner points out that "both Tsarnaevs’ images exist in official databases"..the software failed to make a match.
sehugg 14y ago
What's wrong with needing the public's help? The public did a great job in providing critical evidence. Maybe instead of more intrusive surveillance we find more ways for the public to participate. Maybe we uphold the public's right to take pictures and video in "sensitive areas". Etc.

bsenftner 14y ago

My big questions is why were the suspects faces in the database? Why would a 19 year old, never done anything face be in the facial recognition database?

dr_doom 14y ago
The older brother had been questioned by the FBI before. http://www.cbsnews.com/8301-201_162-57580534/fbi-interviewed-dead-boston-bombing-suspect-years-ago/ http://www.cbsnews.com/8301-201_162-57580534/fbi-interviewed...
DannyBee 14y ago
Best random guess: Something related to him being an alien and becoming a naturalized citizen
- DannyBee 14y ago
  Actually, figured it out: Dzhokhar had a Massachusetts driver’s license, and those pictures go into a facial recognition database. Source: http://www.washingtonpost.com/world/national-security/inside-the-investigation-of-the-boston-marathon-bombing/2013/04/20/19d8c322-a8ff-11e2-b029-8fb7e977ef71_print.html http://www.washingtonpost.com/world/national-security/inside...
sp332 14y ago
He's 19, so he likely has a driver's license or other state-issued ID photo on file.
ics 14y ago
I'm going to venture to guess that either a) the officer was talking out of his ass to make a quick point or b) someone, somewhere massively scraped every social network they could think of for anyone with ties to the area or the event. That being said, I don't really like the sound of it either, regardless of how true it is. --pre-submit thought: perhaps it was the older brother? Our background is rather limited but it would not be surprising if he had had prior run-ins with the law. Even a minor offense will get your mugshot in the file.

mrkmcknz 14y ago

Does anyone have detailed knowledge of startups in this space? I know of Lambda labs but I personally haven't use their solution post Face.com.

brk 14y ago
There are a few, though less than you might think. Lots of general progress has been made in terms of facial detection algorithms, but there are almost no "general" security camera deployments that are setup to capture sufficient detail to get a good face match. There are not enough events like this in open places that drive requirements for facial recognition use. Limited cases (airports, casinos, etc.) more frequently have cameras setup to get good straight-on high-res facial shots, but almost no typical outdoor area is covered that way.

jhonovich 14y ago

I am the author of the referenced article. The problem is matching surveillance camera images (like from Lord & Taylor's) is impossible because of the steep down tilt angle of the cameras plus their low resolution. Smartphone images could be a little better as the angle is more head on and the resolution is higher. However, even those images tended to be in a wide field of view which means low pixel density, hurting matching performance. Last, but not least, the caps and sunglasses hurt recognition as well.

xfs 14y ago
Matching surveillance camera images is always possible no matter how bad the image quality is. The only matter is how many false positives show up. Facial recognition algorithms always give their best matching candidates, with thresholds to skip those not quite matching ones. Your article makes it sound like no such results were produced at all, which is unlikely how recognition is supposed to work.
- onli 14y ago
  Not impossible though. Given that such software is used to steer the future efforts of the investigators, it is possible that the software does say "no match found" if the probability of a real match is too low.
- jhonovich 14y ago
  First of all, almost any facial surveillance system would simply reject images like those from Lord & Taylor's. The pixel density is simply too low to even attempt plus the downtilt would also likely trigger a rejection. That's how facial recognition does work in practice. Even if the system would allow it, not only would false positives be enormous, you'd probably miss the match as well (false negative).
akennberg 14y ago
Interesting to see articles promoting adding more surveillance cameras for all their flaws (bad angle, resolution trails behind phone cameras, etc). I would focus on acquiring publicly available photo data in real time at dense events and scan them. I.e. crowdsourced surveillance. I'm sure someone is working on that.

marknutter 14y ago

How hard would it be to throw off facial recognition by simply wearing very high quality makeup and prosthetics to alter your facial appearance (ala hollywood movies)? If I were the boston bombers, I would have first learned how to make myself look completely different. It's unbelievable to me that these guys didn't realize they'd be captured on tons of cameras that day.

nwh 14y ago
If you wore Mission Impossible style masks, there's no reason why the facial recognition would be able to pick you up from underneath. You might alert the suspicion of people around you more though, as they don't ever seem to be perfect.
- runarb 14y ago
  What if you wore something like this: http://www.spfxmasks.com/ourmasks.php http://www.spfxmasks.com/ourmasks.php ? One should be able to blend in, and make video surveillance pretty useless... Edit: Did work for this robbers: http://www.dailymail.co.uk/news/article-2192115/Robbers-disguised-white-cops-caught--send-polite-thank-letter-company-unbelievable-latex-masks.html http://www.dailymail.co.uk/news/article-2192115/Robbers-disg...
Plimsoll 14y ago
This brings to mind this bank robbery news from few years aback: http://gizmodo.com/5711252/white-guy-becomes-black-bank-robber-with-super-realistic-hollywood-mask http://gizmodo.com/5711252/white-guy-becomes-black-bank-robb... Don't know how easy it is to get those masks though.
speednoise 14y ago
Easier than you may think: http://cvdazzle.com/ http://cvdazzle.com/
gcr 14y ago
CV-Dazzle attempts to throw off face detectors by doing just that: http://cvdazzle.com/ http://cvdazzle.com/ But then again, the police will label you as "that guy with weird bits of plastic on his face", which won't help you hide at all.

jere 14y ago

I watched this lecture recently http://youtu.be/-IUe3ce_29I http://youtu.be/-IUe3ce_29I which talked about how hard it is to do image recognition at low resolution. One solution is a "fovea" system that scans an area for interesting items and zooms in on them with a high resolution camera automatically to get better recognition. I wonder if this would help in similar instances. A system that uses facial detection to isolate faces and then zoom in on them very quickly seems feasible. I'm assuming it would be overwhelmed in a scenario like a marathon of course, but perhaps it would help in other situations.

eksith 14y ago

Bottom line is that it's nearly impossible to exactly match facial features to anyone beyond a set distance when there's too much ambiguity between distances and rubbish quality CCTV video. Matching gait may be a more effective means of identification at a distance (and, in this and many other cases, at odd angles). http://www.am.sanken.osaka-u.ac.jp/~sagawa/pdf/accv06.pdf http://www.am.sanken.osaka-u.ac.jp/~sagawa/pdf/accv06.pdf (PDF)

jpalomaki 14y ago

I can understand that recording live footage with very high quality is a problem, but how about capturing quality stills to complement the video with lower quality.

emiliobumachar 14y ago
Someone with knowledge of cameras please either correct or endorse this, but I think the largest price/quality tradeoff in any camera happens at the lens choice, and given a cheap lens, the stills can't be much better than the video.
- securingsincity 14y ago
  certainly the lens and sensor are the biggest trade offs with price these would have the biggest impact with image clarity and the amount of light taken in to take the image.
- tripzilch 14y ago
  I think that is right (I can't claim much specific knowledge on camera lenses though), it used to be especially bad with the whole "megapixel wars" a few years back. Pixel resolution increased, but image clarity didn't, and you would get much better images on old low-megapixel cameras with relatively high-end lenses than with a cheap but new camera of approx 4-8x the resolution in pixels. However, nowadays I am quite amazed by the picture quality of the average smartphone camera. So maybe something changed? Still, we're talking about better lenses/quality than smartphone photos. They may be sharp, but that comes from the advantage of being up close and coming in huge numbers. And in addition to the lens, there's a lot of other expensive parts in a security system, that may need to be replaced or reconfigured because it is all integrated with the current low-res cameras. The bigger such a system is, the more expensive it becomes to change it all up when better tech has come around, even if the new parts are relatively cheap by themselves.
- brk 14y ago
  Not exactly. A lot of it has to do with the fact that most security cameras are not setup for capture of facial details (or license plates for that matter). Most security cameras are setup for general overview shots, with around 10-20ppf (pixels per foot), whereas good detail starts at around 80ppf (meaning that you'd need 4x as many cameras in most areas to start to get something approaching an image quality that you could use for reliable facial recognition. On top of that there is the storage factor. A "common" camera in a new deployment is a 1080p (2.1 Megapixel) camera. If this is set to record good quality images 24/7, it consumes about 60GB of storage per day. Various tricks like "record on motion", etc. can reduce this to maybe 10GB/day, at the risk of potentially missing key evidence or having gaps in the recording. Lenses are important, but it's not that hard to find a lens with sufficient resolution for these cameras. A good lens rated at 2+ megapixels is around $70-$120 (depending on various factors), and a good outdoor camera (including lens) can be had for $500-$1200. Big price ranges, yes, but not to the point that the lenses are the limiting factor budget-wise.

mikecane 14y ago

How to camouflage yourself from facial recognition technology http://venturebeat.com/2010/07/02/facial-recognition-camouflage/ http://venturebeat.com/2010/07/02/facial-recognition-camoufl...

anigbrowl 14y ago
That's not conspicuous at all. And like zebras, there's no benefit unless everyone else is doing it.
- okamiueru 14y ago
  Dazzle camouflage isn't meant to hide you in your surroundings, but rather make it more difficult to assess the geometry -- how large you are, where you are heading, etc. It's more for confusing the lion running after the zebra, than an attempt hide from the lion.

ck2 14y ago

Not just facial recognition, the "terror watch list" failed (after a warning from Russia about the older brother) lack of background checks on the AR15s they owned failed, the secured perimeter failed (the teenager was outside of it) the city lockdown failed (the guy smoking the cigarette outside when he saw the blood was technically violating the lockdown). In the end it was mostly dumb luck, dumb criminals, and the fact the poor victim who lost their legs managed to survive. In fact that was the success, that more wounded people didn't die thanks to fantastic work by the hospital and doctors. Instead of patting law enforcement on their backs, how about thanking the doctors, where is their applause?

afarrell 14y ago
Actually, the lockdown was voluntary, though with strong community pressure to comply. But yes, multiple subsystems failed in the police response. If you listened to police scanners, you also heard that they had some trouble identifying streets correctly, or keeping their mics off when not talking. But police response protocols are one human system that definitely needs to be able to work in the face of multiple subsystem failure. I say judge the police on how the response worked as a whole. I have to say though, thank god that Dunkin' Doughnuts' Tactical Ration Deployment Vehicle ran smoothly.
- ck2 14y ago
  Talking about the scanners and technology (and failures) didn't "homeland security" give millions to metro police departments around the country to get their communication encrypted? I vaguely remember reading about that a couple years ago. I wonder where that money went if that is accurate. https://www.google.com/search?q=%22federal+grant%22+%22encrypted+radios%22 https://www.google.com/search?q=%22federal+grant%22+%22encry...
  - datr 14y ago
    I've wanted to ask about this since seeing the Boston police channel available on the internet. In the UK all the channels are encrypted making it virtually impossible for a civilian to listen in. Is it simply financial reasons that the USA doesn't do this?

astar 14y ago

Facial recognition is still treated like some magical technology, like the "zoom in, enhance" feature much mocked about on TV crime shows. I bet Facebook's technology is ahead of law enforcement's in terms of data, algorithm, and refinement, and yet it still stumbles on pretty obvious faces (if you have the default auto-tag feature left on)

brk 14y ago
Correct, and you're talking about cases where you might have literally 500,000+ pixels on a face (close-up shot from an 8MP camera). The average cctv shot will have around 200-1000 pixels on the entire face, if you're lucky.
gcr 14y ago
Facebook only has to distinguish between friends of the uploader. US law enforcement has to distinguish between 300 million civilians. Of the two, I would expect Facebook to be more successful at identifying peolpe.
anigbrowl 14y ago
"zoom in, enhance" feature much mocked about on TV crime shows Actually, those jokes refer to a scene in Blade Runner: https://www.youtube.com/watch?v=qHepKd38pr0 https://www.youtube.com/watch?v=qHepKd38pr0
- socillion 14y ago
  NTSF:SD:SUV Parody of the 'enhance' trope in cop drama setting: https://www.youtube.com/watch?v=xbCWYm7B_B4&t=58s https://www.youtube.com/watch?v=xbCWYm7B_B4&t=58s

WestCoastJustin 14y ago

Why did it fail? That, to me, is the most interesting unanswered question.

new_test 14y ago
Because facial recognition is a very difficult problem.
- sliverstorm 14y ago
  Surely all you need to do is start an agile startup and throw some Ruby on Rails at the problem?

gcr 14y ago

The problem is that any non-perfect automated system will wrongly convict so many civilians as being terrorists that nobody will trust it anymore. Let's say that you want to catch terrorists creeping around in your airport. To do this, you have a magic camera that, when it takes a picture of a civilian, compares it to a picture of a known terrorist. If they're in fact the same person, it will flag security over. Let's say that it is indeed a magic camera and makes the correct decision 99.99% of the time, which is far better than the current state of the art[1]. Do you have any idea how much DHS or DoD would pay for such a magic camera? But it won't be enough. Consider a watch list of 100 terrorists that we're looking for in our airport. If our camera compares each civilian's picture to all 100 terrorists in succession, and if we model each comparison as a Bernoulli trial, then each civilian has a 0.9999^100 = 99.004% chance of getting through security unfazed, which means we flag innocent people as terrorists about 1% of the time. This doesn't work in an airport setting. Assuming 5,000 people pass through your small airport per hour and you have a 14-hour day, that means your security people have to go through 0.01 * 5000 * 14 = 700 false alarms each day. That's about 1.2 false alarms per minute. Your security people will have to go through all their checks each time this happens. Before long, they're going to stop trusting your magic camera. As you scale your watch list up, it gets worse. How well would it work for 1,000 people on the watch list? 10,000? Note that the current US watch list has 50,000 people on it. Even if it were a perfect magic camera, it coudln't catch terrorists on their maiden crime who aren't on the watch list at all. (Most of this post was adapted from: http://vast.uccs.edu/~tboult/tmp/IJCB11-tutorial-boult.ppt http://vast.uccs.edu/~tboult/tmp/IJCB11-tutorial-boult.ppt , which is a really interesting presentation if you're interested in these sorts of things.) 1: http://vis-www.cs.umass.edu/lfw/results.html http://vis-www.cs.umass.edu/lfw/results.html

stretchwithme 14y ago
Do you know how unreliably people recognize strangers? Many people are already wrongly convicted on such evidence. Even perfect automated facial recognition won't convict people. It will merely point out people that should be looked at. If a jury looking at the image and the guy's face think the system is wrong, they can disregard it. As far as prevention goes, what we need are systems that can detect deception reliably, which will help if we ever decide to ask people meaningful questions. Not to convict, but to focus attention on a possible threat.