6 ms·

A minor technical nitpick: the article, and the activists it quotes, make the claim that facial recognition discriminates against minorities because it is inacc

by fwlr 3y ago

A minor technical nitpick: the article, and the activists it quotes, make the claim that facial recognition discriminates against minorities because it is inaccurate. The state of the art in facial recognition technology has been superhumanly accurate (e.g. telling apart identical twins better than humans can, finding pictures with significant weight or age differences) on all subgroups for several years now, and the racial disparities in accuracy are minuscule relative to the level of noise. It will keep improving: e.g. it is quite plausible we will be able to infer parts of a person’s genotype from ordinary pictures of their face. I call this a nitpick because “face rec discriminates because it is inaccurate” vs “face rec discriminates because it very accurately reproduces society’s bias” doesn’t make much difference to the activists’ argument; after all, it matters little whether the discrimination arises by bumbling incompetence or by efficient indifference. But I thought I would mention it so that the more technically-minded are not accidentally misled about the technological progress in the area.

lesuorac 3y ago
Sure and state-of-the-art cars will hit the brake before running into another car. Doesn't mean the car behind you at the red light will brake. Just like the fact that a model can tell twins apart doesn't mean the model the cops used can tell you apart from the actual criminal. Or that a Landlord will have tested the model at all or required it to meet some standard of twin detection.
- fwlr 3y ago
  My usage of “state of the art” was misleading, I apologize. To be more precise I should say “most of the companies offering services in this area are this good and getting better”. This is not like batteries or carbon nanotubes where research is way ahead of production; it is more like LLMs where commercial offerings are significantly ahead of published research offerings.
  - jamiek88 3y ago
    Can you give some examples of real world successes of this? Because your (several) comments defending this technology boil down to ‘trust me bro, I’ve seen it in the lab’.
    - waboremo 3y ago
      Why do you think they're "defending this technology" when their comment explicitly highlights how this doesn't go against the activist's core point, only a nitpick of a specific argument about "technical progress"?
    - fwlr 3y ago
      If I’ve given the impression that I am defending the technology against anything other than claims about its accuracy, I apologize. The fact that I used to work on this technology and now I instead talk about how it harms minorities should give you some idea of how likely I am to defend this technology. Basically you should just imagine there was a media cycle about the dangers of unregulated trains that go 50mph, do not have any brakes, and need to be regulated because they’re running pedestrians over. And you should read my comments like I’m a pedantic trainwatcher saying “actually these trains go 400mph, can stop on a dime, and need to be regulated because they’re running pedestrians over. Source: I used to be a train driver”.
Timshel 3y ago
To nitpick :) : SOTA is nice but there is often a wide gap between validation dataset and real world usage accuracy. Additionnaly accuracy percentage are often misleading something over 99% sounds great until it's applied thousands of times and you end up with dozens of failure and if those failure are not gracefully handled it can quickly become a nightmare. And I highly doubt that a "penny-pinching landlord" will install anything close to Sota.
TheRealPomax 3y ago
When was the last time you saw state of the art being applied to a cheap surveillance camera? These aren't video production studios, these are "landlords", which aren't even people, they are housing corporations who spring for the cheapest shit that lets them maximize income and if you don't like it, for you 10,000 other renters willing to put up with their BS to live in NYC.
- fwlr 3y ago
  Facial recognition is an algorithm applied to ordinary picture and video data, it does not require special or even particularly high-quality cameras. In fact, if the footage is too high-quality it actually has to be downscaled before being analyzed; something like a picture of your face taken with a 720p laptop webcam when sitting at the laptop, or a video of your face from a 1080p Ring doorbell camera as you go to ring the bell, would likely be already big enough to require downscaling first.
  - dingusdew 3y ago
    [dead]
  - TheRealPomax 3y ago
    Yep, typically an algorithm with heavily baked in bias. Works great for specific populations, not so much when it needs to recognize every possible facial structure on the planet - something that's a problem in very few places, except that NY is basically #1 on the list of places where you algorithm has to work for every possible face on the planet, and would have to be tamper proof, which by definition it can't be unless we augment it with IR and lidar to make sure there's a human standing in front of it. And then that would have to still work when it's both 100F and -40F. You know what lets people into your building even more efficiently? A key fob.
    - fwlr 3y ago
      Addressing this misconception is the whole point of my original comment, actually. Typically the algorithms used work great on all populations and do not have heavily baked in bias (in the technical sense; they do still have bias in the social justice sense of “automating the detection of criminals, who are disproportionally minorities because society’s bias has disproportionately criminalized minorities”). The field has generally moved away from specific algorithms that analyze facial structure and towards much more effective machine-learning-augmented methods (they got Bitter-Lessoned just like other areas of technology did). Facial recognition has literally already been used to great effect in New York for years already, that’s why it is New York that is leading the way in regulating face rec, because they interact with it the most (apart from maybe Australians at Australian airports). About the only thing I don’t take issue with in your post is LIDAR/IR to verify it’s a real human and not a piece of paper with a face printed on it. If you’re using face rec to automate entry, yeah, you need to also validate it’s a real human face.
bastawhiz 3y ago
I have no faith that landlords will be using the state of the art. Especially when the not-state-of-the-art version supports (in a plausibly deniable way) the biases of the landlords. Make no mistake: landlords are absolutely not doing this out of the goodness of their hearts to make their buildings safer.
afavour 3y ago
Is the state of the art facial recognition technology deployed in real world, scenarios, though? It's one thing to use it in a lab setting with a person sitting still the perfect distance away from the camera, and quite another to operate on hundreds of people at once from a distant ceiling. Not to mention cost (i.e. the landlords don't actually care about accuracy that much, would they really pay a premium?)
nerdix 3y ago
Where are these state of the art facial recognition systems deployed? Because as someone with dark skin and a Nest doorbell with facial recognition, I can tell you from experience that Nest still struggles with dark skin. And if Nest is not deploying state of the art facial recognition then what are the odds that some random landlord is? I'm going to say close to none.
- harvey9 3y ago
  Are there any light-skinned people set up on your doorbell for comparison? Would help to eliminate other possibilities like low light, or the algorithm being generally bad.
  - nerdix 3y ago
    Yes, in fact. I have people in my extended family who are fair skinned (i.e. white) and have visited my home. The Nest was almost immediately able to accurately identity them even in sub-optimal lighting conditions. Even though it struggles to identity me except in the most optimal lighting conditions. Despite the fact that it "sees" me a lot more. It was one of the first things I noticed when I got the doorbell. The difference was stark and obvious.
- fwlr 3y ago
  No, I do agree that Nest’s face rec (and face detection too, for that matter) is pretty awful. I don’t have first-hand knowledge of what Google is actually deploying but I suspect they’re performing so poorly because they’re training the recognition on just the home’s dataset; it is sort of like each home training it’s own LLM just on text written by that home’s inhabitants, while GPT4 gets unlimited resources to train on hundreds of gigabytes of text from all over the web. But it is still a bit of a mystery why Google is allowing Nest’s facial recognition to remain so weak. Actually, if Nest is representative of what most consumers experience when interacting with facial recognition, then I understand why there’s this common perception that facial recognition is buggy and unreliable. I’m drawing on my experience working on commercial models.
  - 3y ago
mc32 3y ago
Have you considered that the activists might be okay with surveillance if the surveillance is fair? That is, if it's not misidentifying people and adds a sense of security and auditability, maybe they would be okay, just as they themselves might install Nest/Ring in their own abode? Now, I agree that if the argument is "surveillance is inaccurate" then some landlords and landladies might come back and say, "ours is more accurate than trained crowds of professional forensic experts" and it's possible the activists would not want that and just don't want any surveillance, or maybe they are looking for accuracy and the elimination of misattribution.
- fwlr 3y ago
  I have considered that, empirically speaking I’ve seen a few activists do exactly that, and I am definitely not going around believing that activists are all some kind of anti-all-surveillance, defund-the-cops, legalize-crime parody. But also empirically speaking, I have seen that highly accurate systems with no racial bias still disproportionately impact minorities (often because they are employed by institutions like law enforcement or landlords that have racial bias themselves). I really just think that if activists see two stats like “provably 99.999…% accurate” and “provably impacts minorities more” at the same time, most of them are still going to care about the impact than the accuracy - and I don’t think that caring about the impact is in any way intellectually dishonest, it’s quite moral really.
NegativeLatency 3y ago
In practice I suspect we’ll see many more misidentifications like the one that happened recently https://www.theguardian.com/us-news/2023/apr/27/california-police-facial-recognition-software https://www.theguardian.com/us-news/2023/apr/27/california-p...
cm2187 3y ago
I played with AWS rekog, I agree that it is remarkable, but even with a 98 threshold you get occasionally some false positives (but rare), and often some false negatives (I didn't count but I'd say of the magnitude of 0.1-0.5%). That would mean that several days in the year you can't get in because the system mis identifies you.
dmbche 3y ago
https://sitn.hms.harvard.edu/flash/2020/racial-discrimination-in-face-recognition-technology/ https://sitn.hms.harvard.edu/flash/2020/racial-discriminatio... While I'm sure some algorithm is provably unbiased, it's a little naive to imagine landlords (you know about the landlord special? Load bearing paint and the likes?) caring about what they are running.
libraryatnight 3y ago
Just so everyone knows this is an account that just bullshits elaborately. Everything the account says is just for argument sake.
- pepperonipizza 3y ago
  He explains his point convincingly well. You should make your point in reply if you disagree with OP, because slandering OP is not changing my pov to yours here.
  - gusgus01 3y ago
    I'd say the fact that they write-off repeated examples of existing systems that are deployed commercially failing, and then also haven't responded with any real world examples of systems that are "nearly foolproof" as a pretty good indicator of arguing a hypothetical.
  - libraryatnight 3y ago
    No he doesn't. His entire comment history is exactly what he's done here. Decently written but empty and contrarian for the sake of it. Engaging with this person is a waste of time, I was merely providing a public service to others that similarly value their time.
lisasays 3y ago
The state of the art in facial recognition technology has been superhumanly accurate That's SOA and in controlled circumstances you're talking about. Not real-life FR using cheap cameras, with bad lighting, against shitty lookup databases (with built-in biases). Which is what's at issue, here. Your apologetic does make for excellent ad copy & pitch deck material, however.
darth_avocado 3y ago
> The state of the art in facial recognition technology has been superhumanly accurate On a clear pictures, maybe. On a regular blurry surveillance footage from a gas station camera on a day with bad weather and poor lighting, absolutely not.
janalsncm 3y ago
It’s also an extremely weak argument to say that FR is inaccurate. It’s contingent on an engineering problem that will be solved. Algorithms will get better, HD cameras will become cheaper and more ubiquitous. We are rushing head first into a future where every public and semi-public interaction is recorded and analyzed by functionally omniscient systems, and the temporary inaccuracy of that analysis does nothing to argue against that future. And then I ask myself, is that really a bad thing? I am far more likely to be the victim of a crime than the perpetrator of one or the victim of inaccurate accusation. The places I’ve lived with pervasive surveillance were also some of the safest, and those who argue against this “oppressive” surveillance rarely are forced to balance that with the actual oppression of crime, or the second-order consequence of the lack of security cameras (that low level crimes will probably go unpunished). I should also add that going back to a time where surveillance doesn’t exist is not an option. Everyone is walking around with a security camera in their pocket. Very soon those videos will be able to be run through facial recognition tech as well.
- JohnFen 3y ago
  > We are rushing head first into a future where every public and semi-public interaction is recorded and analyzed by functionally omniscient systems, and the temporary inaccuracy of that analysis does nothing to argue against that future. I agree. We need to be fighting the panopticon. The more accurate it is, the more objectionable it is.
dragonwriter 3y ago
> The state of the art in facial recognition technology has been superhumanly accurate (e.g. telling apart identical twins better than humans can, finding pictures with significant weight or age differences) on all subgroups for several years now, and the racial disparities in accuracy are minuscule relative to the level of noise. That may be true of state of the art models that have been demonstrated under some conditions; it continues not to be true, frequently, of models actually adopted in major products, even new ones, in the actual deployment conditions (e.g., the recently adopted CBP One app that US asylum seekers are required to use for appointment scheduling and other purposes, and which requires facial recognition, and disproportionately fails for dark skinned users, who are not uncommon in the population required to use the app.)

by fwlr 3y ago

lesuorac 3y ago

Sure and state-of-the-art cars will hit the brake before running into another car. Doesn't mean the car behind you at the red light will brake. Just like the fact that a model can tell twins apart doesn't mean the model the cops used can tell you apart from the actual criminal. Or that a Landlord will have tested the model at all or required it to meet some standard of twin detection.

fwlr 3y ago
My usage of “state of the art” was misleading, I apologize. To be more precise I should say “most of the companies offering services in this area are this good and getting better”. This is not like batteries or carbon nanotubes where research is way ahead of production; it is more like LLMs where commercial offerings are significantly ahead of published research offerings.
- jamiek88 3y ago
  Can you give some examples of real world successes of this? Because your (several) comments defending this technology boil down to ‘trust me bro, I’ve seen it in the lab’.
  - waboremo 3y ago
    Why do you think they're "defending this technology" when their comment explicitly highlights how this doesn't go against the activist's core point, only a nitpick of a specific argument about "technical progress"?
  - fwlr 3y ago
    If I’ve given the impression that I am defending the technology against anything other than claims about its accuracy, I apologize. The fact that I used to work on this technology and now I instead talk about how it harms minorities should give you some idea of how likely I am to defend this technology. Basically you should just imagine there was a media cycle about the dangers of unregulated trains that go 50mph, do not have any brakes, and need to be regulated because they’re running pedestrians over. And you should read my comments like I’m a pedantic trainwatcher saying “actually these trains go 400mph, can stop on a dime, and need to be regulated because they’re running pedestrians over. Source: I used to be a train driver”.

Timshel 3y ago

To nitpick :) : SOTA is nice but there is often a wide gap between validation dataset and real world usage accuracy. Additionnaly accuracy percentage are often misleading something over 99% sounds great until it's applied thousands of times and you end up with dozens of failure and if those failure are not gracefully handled it can quickly become a nightmare. And I highly doubt that a "penny-pinching landlord" will install anything close to Sota.

TheRealPomax 3y ago

When was the last time you saw state of the art being applied to a cheap surveillance camera? These aren't video production studios, these are "landlords", which aren't even people, they are housing corporations who spring for the cheapest shit that lets them maximize income and if you don't like it, for you 10,000 other renters willing to put up with their BS to live in NYC.

fwlr 3y ago
Facial recognition is an algorithm applied to ordinary picture and video data, it does not require special or even particularly high-quality cameras. In fact, if the footage is too high-quality it actually has to be downscaled before being analyzed; something like a picture of your face taken with a 720p laptop webcam when sitting at the laptop, or a video of your face from a 1080p Ring doorbell camera as you go to ring the bell, would likely be already big enough to require downscaling first.
- dingusdew 3y ago
  [dead]
- TheRealPomax 3y ago
  Yep, typically an algorithm with heavily baked in bias. Works great for specific populations, not so much when it needs to recognize every possible facial structure on the planet - something that's a problem in very few places, except that NY is basically #1 on the list of places where you algorithm has to work for every possible face on the planet, and would have to be tamper proof, which by definition it can't be unless we augment it with IR and lidar to make sure there's a human standing in front of it. And then that would have to still work when it's both 100F and -40F. You know what lets people into your building even more efficiently? A key fob.
  - fwlr 3y ago
    Addressing this misconception is the whole point of my original comment, actually. Typically the algorithms used work great on all populations and do not have heavily baked in bias (in the technical sense; they do still have bias in the social justice sense of “automating the detection of criminals, who are disproportionally minorities because society’s bias has disproportionately criminalized minorities”). The field has generally moved away from specific algorithms that analyze facial structure and towards much more effective machine-learning-augmented methods (they got Bitter-Lessoned just like other areas of technology did). Facial recognition has literally already been used to great effect in New York for years already, that’s why it is New York that is leading the way in regulating face rec, because they interact with it the most (apart from maybe Australians at Australian airports). About the only thing I don’t take issue with in your post is LIDAR/IR to verify it’s a real human and not a piece of paper with a face printed on it. If you’re using face rec to automate entry, yeah, you need to also validate it’s a real human face.

bastawhiz 3y ago

I have no faith that landlords will be using the state of the art. Especially when the not-state-of-the-art version supports (in a plausibly deniable way) the biases of the landlords. Make no mistake: landlords are absolutely not doing this out of the goodness of their hearts to make their buildings safer.

afavour 3y ago

Is the state of the art facial recognition technology deployed in real world, scenarios, though? It's one thing to use it in a lab setting with a person sitting still the perfect distance away from the camera, and quite another to operate on hundreds of people at once from a distant ceiling. Not to mention cost (i.e. the landlords don't actually care about accuracy that much, would they really pay a premium?)

nerdix 3y ago

Where are these state of the art facial recognition systems deployed? Because as someone with dark skin and a Nest doorbell with facial recognition, I can tell you from experience that Nest still struggles with dark skin. And if Nest is not deploying state of the art facial recognition then what are the odds that some random landlord is? I'm going to say close to none.

harvey9 3y ago
Are there any light-skinned people set up on your doorbell for comparison? Would help to eliminate other possibilities like low light, or the algorithm being generally bad.
- nerdix 3y ago
  Yes, in fact. I have people in my extended family who are fair skinned (i.e. white) and have visited my home. The Nest was almost immediately able to accurately identity them even in sub-optimal lighting conditions. Even though it struggles to identity me except in the most optimal lighting conditions. Despite the fact that it "sees" me a lot more. It was one of the first things I noticed when I got the doorbell. The difference was stark and obvious.
fwlr 3y ago
No, I do agree that Nest’s face rec (and face detection too, for that matter) is pretty awful. I don’t have first-hand knowledge of what Google is actually deploying but I suspect they’re performing so poorly because they’re training the recognition on just the home’s dataset; it is sort of like each home training it’s own LLM just on text written by that home’s inhabitants, while GPT4 gets unlimited resources to train on hundreds of gigabytes of text from all over the web. But it is still a bit of a mystery why Google is allowing Nest’s facial recognition to remain so weak. Actually, if Nest is representative of what most consumers experience when interacting with facial recognition, then I understand why there’s this common perception that facial recognition is buggy and unreliable. I’m drawing on my experience working on commercial models.
- 3y ago

mc32 3y ago

Have you considered that the activists might be okay with surveillance if the surveillance is fair? That is, if it's not misidentifying people and adds a sense of security and auditability, maybe they would be okay, just as they themselves might install Nest/Ring in their own abode? Now, I agree that if the argument is "surveillance is inaccurate" then some landlords and landladies might come back and say, "ours is more accurate than trained crowds of professional forensic experts" and it's possible the activists would not want that and just don't want any surveillance, or maybe they are looking for accuracy and the elimination of misattribution.

fwlr 3y ago
I have considered that, empirically speaking I’ve seen a few activists do exactly that, and I am definitely not going around believing that activists are all some kind of anti-all-surveillance, defund-the-cops, legalize-crime parody. But also empirically speaking, I have seen that highly accurate systems with no racial bias still disproportionately impact minorities (often because they are employed by institutions like law enforcement or landlords that have racial bias themselves). I really just think that if activists see two stats like “provably 99.999…% accurate” and “provably impacts minorities more” at the same time, most of them are still going to care about the impact than the accuracy - and I don’t think that caring about the impact is in any way intellectually dishonest, it’s quite moral really.

NegativeLatency 3y ago

In practice I suspect we’ll see many more misidentifications like the one that happened recently https://www.theguardian.com/us-news/2023/apr/27/california-police-facial-recognition-software https://www.theguardian.com/us-news/2023/apr/27/california-p...

cm2187 3y ago

I played with AWS rekog, I agree that it is remarkable, but even with a 98 threshold you get occasionally some false positives (but rare), and often some false negatives (I didn't count but I'd say of the magnitude of 0.1-0.5%). That would mean that several days in the year you can't get in because the system mis identifies you.

dmbche 3y ago

https://sitn.hms.harvard.edu/flash/2020/racial-discrimination-in-face-recognition-technology/ https://sitn.hms.harvard.edu/flash/2020/racial-discriminatio... While I'm sure some algorithm is provably unbiased, it's a little naive to imagine landlords (you know about the landlord special? Load bearing paint and the likes?) caring about what they are running.

libraryatnight 3y ago

Just so everyone knows this is an account that just bullshits elaborately. Everything the account says is just for argument sake.

pepperonipizza 3y ago
He explains his point convincingly well. You should make your point in reply if you disagree with OP, because slandering OP is not changing my pov to yours here.
- gusgus01 3y ago
  I'd say the fact that they write-off repeated examples of existing systems that are deployed commercially failing, and then also haven't responded with any real world examples of systems that are "nearly foolproof" as a pretty good indicator of arguing a hypothetical.
- libraryatnight 3y ago
  No he doesn't. His entire comment history is exactly what he's done here. Decently written but empty and contrarian for the sake of it. Engaging with this person is a waste of time, I was merely providing a public service to others that similarly value their time.

lisasays 3y ago

The state of the art in facial recognition technology has been superhumanly accurate That's SOA and in controlled circumstances you're talking about. Not real-life FR using cheap cameras, with bad lighting, against shitty lookup databases (with built-in biases). Which is what's at issue, here. Your apologetic does make for excellent ad copy & pitch deck material, however.

darth_avocado 3y ago

> The state of the art in facial recognition technology has been superhumanly accurate On a clear pictures, maybe. On a regular blurry surveillance footage from a gas station camera on a day with bad weather and poor lighting, absolutely not.

janalsncm 3y ago

It’s also an extremely weak argument to say that FR is inaccurate. It’s contingent on an engineering problem that will be solved. Algorithms will get better, HD cameras will become cheaper and more ubiquitous. We are rushing head first into a future where every public and semi-public interaction is recorded and analyzed by functionally omniscient systems, and the temporary inaccuracy of that analysis does nothing to argue against that future. And then I ask myself, is that really a bad thing? I am far more likely to be the victim of a crime than the perpetrator of one or the victim of inaccurate accusation. The places I’ve lived with pervasive surveillance were also some of the safest, and those who argue against this “oppressive” surveillance rarely are forced to balance that with the actual oppression of crime, or the second-order consequence of the lack of security cameras (that low level crimes will probably go unpunished). I should also add that going back to a time where surveillance doesn’t exist is not an option. Everyone is walking around with a security camera in their pocket. Very soon those videos will be able to be run through facial recognition tech as well.

JohnFen 3y ago
> We are rushing head first into a future where every public and semi-public interaction is recorded and analyzed by functionally omniscient systems, and the temporary inaccuracy of that analysis does nothing to argue against that future. I agree. We need to be fighting the panopticon. The more accurate it is, the more objectionable it is.

dragonwriter 3y ago

> The state of the art in facial recognition technology has been superhumanly accurate (e.g. telling apart identical twins better than humans can, finding pictures with significant weight or age differences) on all subgroups for several years now, and the racial disparities in accuracy are minuscule relative to the level of noise. That may be true of state of the art models that have been demonstrated under some conditions; it continues not to be true, frequently, of models actually adopted in major products, even new ones, in the actual deployment conditions (e.g., the recently adopted CBP One app that US asylum seekers are required to use for appointment scheduling and other purposes, and which requires facial recognition, and disproportionately fails for dark skinned users, who are not uncommon in the population required to use the app.)