11 ms·

I will say that 3 years ago when we tested all the major vendors' image recognition technology, they all failed spectacularly. And this wasn't some small projec

I will say that 3 years ago when we tested all the major vendors' image recognition technology, they all failed spectacularly. And this wasn't some small project, we ran tens of thousands of our images through (out of over a million) and worked with several Phd students in our efforts. A picture with a white college kid in the frame would get the output of "human". Put an African borrower in the frame and you got at best a failure to recognize a human, and at worst a reference to an animal. I would hope the situation is much better now, but the bias (and just sheer inaccuracy) of the tools was readily apparent and we gave up on image recognition for the time being.

formercoder 6y ago
There are real technical challenges here. Lighter skin has more contrast between light and dark.
- gugagore 6y ago
  Worth noting that sometimes technical challenges may arise because of the way technology was designed as opposed to being inherently challenging https://www.npr.org/2014/11/13/363517842/for-decades-kodak-s-shirley-cards-set-photography-s-skin-tone-standard https://www.npr.org/2014/11/13/363517842/for-decades-kodak-s... but, yes, dark skin reflects less light than light skin. by definition.
- ponker 6y ago
  But the fact that the technology is considered viable given this failure mode is what reflects the bias. If a vendor has tech that can only distinguish white faces it will sell it as “detect your best customers and offer them special deals when they enter your store.” If the tech can only recognize black faces it will be sold as “recognize known criminals in the neighborhood and alert police when they enter your store.”
  - nostromo 6y ago
    I'd like to know what cartoon villain tech company you're referring to...
- centimeter 6y ago
  White people also have higher facial variance in general. I vaguely remember in university we had an assignment to generate “eigenfaces” or something and if you partitioned the faces by race the output of the SVD would be much wider for white people. This isn’t especially surprising when you consider the fact that white people have more light/dark contrast, more hair colors, more eye colors, etc. I think a lot of the “bias” complaints levied against algorithms like this are not bias at all, but just humans who are unhappy the world doesn’t quite live up to their idealized model.
  - rockarage 6y ago
    When you sample from a smaller pool you will make uninformed statements like this. Black/Dark people of the world are not limited to Black people in Atlanta. Bet many people here do not know that there are Black people in this world, some who have naturally blonde hair and some who have blue eyes, google it. Moreover capturing Black/Dark skin and features requires more accurate light metering & lighting because dark skin absorb more light. There's a lot variance in cheekbones, nose and lips. Humans' features in general, are more complicated then you realize.
    - throwaway391003 6y ago
      Ok but that can still be true (blonde hair, blue eyes) while there still being much more variations in the white population than in the black population. I'm curious how many white people there are on earth vs how many black people there are, and other races. A couple google searches didn't give me any easy finds
      - user982 6y ago
        > Ok but that can still be true (blonde hair, blue eyes) while there still being much more variations in the white population than in the black population. Even the man who coined "Caucasian" as a racial category recognized that there was more physical variance among African populations and individuals than compared with Europeans.
- chillacy 6y ago
  I recall being amused maybe 3 years ago when I took a picture of two friends, one with dark skin, one with light skin, and the camera choose to expose for the darker skin. He looked fine, the lighter skin friend looked like snow lit by headlights. Completely washed out. The camera settings and background lighting make a big difference.
  - andromeduck 6y ago
    Was this on a DSLR/MILC? My Sony seems to do this sometimes when I have a very dark and very light subject outside but it's usually fine in RAW.
    - mschuster91 6y ago
      On my A7S2 I simply use the exposure correction wheel in such situations - or use the exposure series function so I can mess around with HDR later in Photoshop.
      - andromeduck 6y ago
        Yeah I usually just use exposure comp too, it works pretty well most of the time but candids when everyone's moving and the background is changing can be pretty hard without a fill flash to help out a bit. Thank god for iso invariance though.
  - dingaling 6y ago
    Which is why photographers ignore autoexposure and instead meter on a known reference such as an 18% grey card, or green grass.
    - WorldMaker 6y ago
      I learned that one convention group I'm a part of converged on very visible glossy white badge cards not just because it is easy to sharpie names visibly on (in classic "Hello my name is" style), but also out of a request from the photography group as it is a very useful meter reference visible in almost every photo of any group member. Seeing how many great photos come out of the event in sometimes very strange lighting conditions (as you might expect from an event that includes everything from brightly lit outdoors events to indoor conference spaces and theaters to rock show-lit concerts, and everything in between), I'd imagine that little metering trick is doing wonders.
- aeturnum 6y ago
  There are real technical challenges in producing photographs of both light and dark skin, but the focus on light skinned subjects have led to an accumulation of technical solutions. Computerized facial recognition has been progressing for something like 50 years, with each paper building on previous work. I suspect a parallel world where the dominant social group hard dark skin would explore different technical avenues, focus on ways of recognizing faces that lean less on tone, etc. Obviously it's hard to say with what we know now. Humans have difficulty recognizing cross-ethnic faces, however I don't think dark skinned people are worse at recognizing same-ethnic faces than light skinned people. To me that suggests our social history is more to blame than some fundimental quality of skin tone.
- mwcampbell 6y ago
  But does the reason matter when the result is discrimination against people through no fault of their own?
  - raxxorrax 6y ago
    Please discriminate against me too if that means facial recognition isn't working.
    - krisoft 6y ago
      That’s not necessarily what you want. “not working” can manifest itself in false positive matches too. Cue “all X people look the same” stupidness. And the truth is, the danger zone is when the algo is somewhat ok, but a few percents worse on the discriminated group. If it were obviously stupidly bad (like for example any two black people are a positive match according to it) then people will disregard it. But if its just slightly bad thats harder to notice and suddenly your loan/car rental is rejected because the security system says you look too much like a specific known fraudster. And of course they wont even tell you that thats the reason, because security.
      - raxxorrax 6y ago
        I worked in imaging for medical diagnostic support and dark complexions are just plainly more difficult to analyse because of restricted illumination times and limited contrast. Sensors got much better in recent years though. The indications in question were extremely rare for black people on the other hand. Another use case was the detection of Vitiligo, which is significantly more easy on black people. There was a great demand for therapies because patients are stigmatized for different reasons, especially in Africa because the symptoms can look similar to some deadly diseases. There is a lot of discrimination in imaging, but it doesn't have to do with race and more with the creation of binary images. As for facial recognition. I see some use cases for convenience, but it would be far more effective to disallow widespread application because of the numerous unsolved problems. So I see the move from IBM positively, even if they continue their work. In the end I believe face reg to be possible without bias, but I don't have these propped up security needs.
- pfranz 6y ago
  It goes even deeper than that. If you look at the history of photography, film, and television you'll see a bias towards white skin tones that standards have been built up around. Just looking at "leader ladies" or common reference images there isn't a bunch of diversity. In practice I've heard cinematographers and photographers bring it up around filming different ethnicities. It starts with changing the exposure, but means you have to spend more attention towards color reproduction. This can extend beyond the film or image sensor to biasing lighting equipment, defaults, standards, and thresholds. Looking through the literature you'll see things like, "the design decisions in the NTSC’s test images effectively biased the format toward rendering white people as more lifelike than other races" https://sterneworks.org/Mulvin-Sterne-Scenes.pdf https://sterneworks.org/Mulvin-Sterne-Scenes.pdf I'm mostly familiar with the history in the US. I'm earnestly curious about how other countries accommodated these types of things as they adopted the technologies.
  - dnautics 6y ago
    It's deeper than that. The chemicals used in analog color photography didn't have a dye profile that could capture people of color effectively until the 80s.
    - sangnoir 6y ago
      Even deeper- Kodak only started putting effort to get the process right in the 70s after customers complained about poor color reproduction[1] in wood grain and chocolate 1. https://petapixel.com/2015/09/19/heres-a-look-at-how-color-film-was-originally-biased-toward-white-people/ https://petapixel.com/2015/09/19/heres-a-look-at-how-color-f...
    - solarkraft 6y ago
      I suspect it to be deeper than that: The relevant people didn't think it was worth it to invest all that much into developing it.
    - lallysingh 6y ago
      I'm guessing that nobody cared to try.
- TuringNYC 6y ago
  If there are technical challenges which would produced bias results...perhaps the answer is to not release the product. Perhaps do more R&D or more training on broader samples until one has a better product to release?
  - xupybd 6y ago
    I don't think they are releasing products to make the world a better place. After all this tech is very dangerous. I don't the the ethics of racial bias are factored in to the decision making at these companies. This is one bias I think benefits folks with dark skin. I'd love to be in the group that doesn't get recognised by this terrifying tech.
robomartin 6y ago
> but the bias (and just sheer inaccuracy) I would hope that on HN one would take the time to explore the technical issues before even suggesting bias (which implies human racial discrimination of some sort). I am going to venture a guess that there's a large audience in the image processing/AI/ML world that lacks a fundamental understanding of image sensor and lens technology. I have never seen mention of concepts such as well capacity, quantum efficiency, noise floor, dynamic range, thermal noise, gamma encoding, compression induced errors, etc. in most work I have reviewed. Sensors used in the general class of imagers found in these experiments are nowhere near adequate to capture the full dynamics of a lot of real life images. The lowlights (referring to the lower portion of the dynamic range of a camera, encoding, compression and image processing system) can be some of the most challenging portions of the dynamic range to get quality data. The old idea applies: Garbage-in, garbage-out. It should come as no surprise that algorithms trained on (likely) bad images with bad lowlight detail will fail to deal with people of darker skin. It's almost a given. One can't assume cheap cameras and the data sets produced with these cameras will see the world the way our eyes are able to. Not to mention the fact that we have something called "understanding" while classifier systems have no clue whatsoever what they are looking at, all they can do is put things in buckets and that's that. In other words, there is no inherent comprehension of what a human being might be versus a bear or a teapot. That's a major problem. The answer isn't to give up. The answer is to understand and then go back and do it right. This isn't going to be cheap and it will likely require rethinking how we build and train these systems. As a tangentially related data point, I have three German Shepherd dogs. Two are the traditional black and brown coloring. The third is 100% black. It is virtually impossible to take a good picture of him. In anything but the right lighting he shows up as a dark amorphous blob. For all the prowess of the mighty camera in an iPhone 10, you'd be hard pressed to use those images to recognize him as anything other than a blob on a dark couch. I do have access to high performance images with far greater well capacity as well as 100% uncompressed data output. In that case there's usable data in the lowlights that, through gamma and LUT manipulation can be extracted. When you do that he quickly goes from looking like a blob to looking like a happy dog. Anyone interested in learning more, I would highly recommend looking up Jim Janesick: https://www.google.com/search?q=jim+janesick https://www.google.com/search?q=jim+janesick The popular phrase "he wrote the book" applies here. Jim's books on the subject of image sensor technology (science and design of sensors) are the reference work anyone in imaging studies. He designed so many sensors for space applications I am not sure he even remembers how many. I was fortunate enough to study CCD and CMOS sensor design under him a couple of decades ago. ML has to start with good data. Inadequate sensors coupled with compression and other processing artifacts leads to bad data, a formula for failure.
deleted 6y ago
[deleted]
electro_blah 6y ago
There's a simple fix. just hire a few white dudes to follow them around. this could lead to some parking related issues later though.
- LanceH 6y ago
  You can just hire white dudes. You'll need to hire some black dudes as well so there isn't a diversity gap.
- legostormtroopr 6y ago
  Yeah, but thats going to lead to a hiring bias. So half the people you hire should be black, but then you'll need white guys to follow them around. And for those who don't know the reference: https://www.imdb.com/title/tt1346402/ https://www.imdb.com/title/tt1346402/
  - electro_blah 6y ago
    Yes, that's when the parking issue arise.
bilbo0s 6y ago
I've had this issue with AI's in the medical industry. At best, you have AI's that can easily recognize pathologies that an average rad could recognize, but are useless when it comes to trying to recognize a pathology that 99.999% of rads would miss. Why? Probably because the system is biased towards pathologies that rads recognize. Why? Probably because that's what's in the learning set. I understand all that, but that makes the AI almost useless in a production setting simply because of the way many healthcare delivery networks are structured with respect to radiology. At worst, you have AI's that seem to recognize pathologies that an average rad could recognize, but then inexplicably have a horrible miss on an obvious study that any first year radiologist would have caught while half asleep. It's the sort of miss that makes the astute observer wonder if the other vendor's AI, the one that didn't have any misses on your test set, was simply not fed an example study that triggered its blindspot? You start to wonder where the blindspot on that one was? You start to wonder does it even have a blindspot? How can you work around blindspots like this in general? Etc. But here's the thing, it's never good when you're thinking about mitigations while you're still testing the AI. My first RSNA was over 20 years ago. To this day, we're still hearing the same promises, and the production testing, (it never fails), is still uncovering the same issues. Now I would have thought that recognizing a human would be easier than trying to ascertain, say, calcification in a DX, but apparently these same issues crop up in a number of different applications of ML based technologies. I understand that a training set, by the very definition of the word "set", does not contain everything. Obviously, there will be bias towards whatever is in the training set. Essentially, most techniques today, simply train AI's to tell us whatever we tell the AI's to tell us. But for this reason it should be unsurprising that these sorts of AI's work best in settings with well defined domain spaces and are challenged when the domain space is less well defined. (And in either case, these AI's will have an inescapable bias towards whatever was in the training set.) What does a tumor look like? Is probably too broad a question to give these AI's. What does a stop sign look like? Is probably a question that these AI's could answer relatively reliably. (I hope?) You would have thought that "what does a human look like?" would be closer to "what does a stop sign look like?" But I guess it's not surprising to hear that it seems to trend towards "what does a tumor look like?" in practice.
- jogundas 6y ago
jogundas 6y ago
From what I understand, NIST has been running a live "competition" on faces of 9 million people for some time already [0]. They are trying to evaluate racial biases in particular [1]. Was there a reason why you decided to not rely on what NIST was putting out? [0] https://www.nist.gov/programs-projects/face-recognition-vendor-test-frvt https://www.nist.gov/programs-projects/face-recognition-vend... [1] https://www.nist.gov/news-events/news/2019/12/nist-study-evaluates-effects-race-age-sex-face-recognition-software https://www.nist.gov/news-events/news/2019/12/nist-study-eva...
mschuster91 6y ago
> Put an African borrower in the frame and you got at best a failure to recognize a human, and at worst a reference to an animal. That happens uncomfortably often - and not even anywhere where AI is even remotely involved, like soap dispensers: https://reporter.rit.edu/tech/bigotry-encoded-racial-bias-technology https://reporter.rit.edu/tech/bigotry-encoded-racial-bias-te...
ipiz0618 6y ago
Improving the training set is the way to go. Like others have said, maybe we should attempt to improve contrast of darker faces before training (which won't be easy as well). That said, a non-diverse facial dataset in a diverse society like the US is simply useless. It doesn't help saying the AI is suffering from a human bias, and dropping these projects entirely, unless they are being used for a malign purpose like what China does.
userbinator 6y ago
A picture with a white college kid in the frame would get the output of "human". Put an African borrower in the frame and you got at best a failure to recognize a human, and at worst a reference to an animal. I wouldn't call that bias, but instead the pure and unadulterated innocent honesty of a machine doing what it does best: finding a closest match. A (white or otherwise non-African) human with zero knowledge of race and racism would probably give exactly the same result.

formercoder 6y ago

There are real technical challenges here. Lighter skin has more contrast between light and dark.

gugagore 6y ago
Worth noting that sometimes technical challenges may arise because of the way technology was designed as opposed to being inherently challenging https://www.npr.org/2014/11/13/363517842/for-decades-kodak-s-shirley-cards-set-photography-s-skin-tone-standard https://www.npr.org/2014/11/13/363517842/for-decades-kodak-s... but, yes, dark skin reflects less light than light skin. by definition.
ponker 6y ago
But the fact that the technology is considered viable given this failure mode is what reflects the bias. If a vendor has tech that can only distinguish white faces it will sell it as “detect your best customers and offer them special deals when they enter your store.” If the tech can only recognize black faces it will be sold as “recognize known criminals in the neighborhood and alert police when they enter your store.”
- nostromo 6y ago
  I'd like to know what cartoon villain tech company you're referring to...
centimeter 6y ago
White people also have higher facial variance in general. I vaguely remember in university we had an assignment to generate “eigenfaces” or something and if you partitioned the faces by race the output of the SVD would be much wider for white people. This isn’t especially surprising when you consider the fact that white people have more light/dark contrast, more hair colors, more eye colors, etc. I think a lot of the “bias” complaints levied against algorithms like this are not bias at all, but just humans who are unhappy the world doesn’t quite live up to their idealized model.
- rockarage 6y ago
  When you sample from a smaller pool you will make uninformed statements like this. Black/Dark people of the world are not limited to Black people in Atlanta. Bet many people here do not know that there are Black people in this world, some who have naturally blonde hair and some who have blue eyes, google it. Moreover capturing Black/Dark skin and features requires more accurate light metering & lighting because dark skin absorb more light. There's a lot variance in cheekbones, nose and lips. Humans' features in general, are more complicated then you realize.
  - throwaway391003 6y ago
    Ok but that can still be true (blonde hair, blue eyes) while there still being much more variations in the white population than in the black population. I'm curious how many white people there are on earth vs how many black people there are, and other races. A couple google searches didn't give me any easy finds
    - user982 6y ago
      > Ok but that can still be true (blonde hair, blue eyes) while there still being much more variations in the white population than in the black population. Even the man who coined "Caucasian" as a racial category recognized that there was more physical variance among African populations and individuals than compared with Europeans.
chillacy 6y ago
I recall being amused maybe 3 years ago when I took a picture of two friends, one with dark skin, one with light skin, and the camera choose to expose for the darker skin. He looked fine, the lighter skin friend looked like snow lit by headlights. Completely washed out. The camera settings and background lighting make a big difference.
- andromeduck 6y ago
  Was this on a DSLR/MILC? My Sony seems to do this sometimes when I have a very dark and very light subject outside but it's usually fine in RAW.
  - mschuster91 6y ago
    On my A7S2 I simply use the exposure correction wheel in such situations - or use the exposure series function so I can mess around with HDR later in Photoshop.
    - andromeduck 6y ago
      Yeah I usually just use exposure comp too, it works pretty well most of the time but candids when everyone's moving and the background is changing can be pretty hard without a fill flash to help out a bit. Thank god for iso invariance though.
- dingaling 6y ago
  Which is why photographers ignore autoexposure and instead meter on a known reference such as an 18% grey card, or green grass.
  - WorldMaker 6y ago
    I learned that one convention group I'm a part of converged on very visible glossy white badge cards not just because it is easy to sharpie names visibly on (in classic "Hello my name is" style), but also out of a request from the photography group as it is a very useful meter reference visible in almost every photo of any group member. Seeing how many great photos come out of the event in sometimes very strange lighting conditions (as you might expect from an event that includes everything from brightly lit outdoors events to indoor conference spaces and theaters to rock show-lit concerts, and everything in between), I'd imagine that little metering trick is doing wonders.
aeturnum 6y ago
There are real technical challenges in producing photographs of both light and dark skin, but the focus on light skinned subjects have led to an accumulation of technical solutions. Computerized facial recognition has been progressing for something like 50 years, with each paper building on previous work. I suspect a parallel world where the dominant social group hard dark skin would explore different technical avenues, focus on ways of recognizing faces that lean less on tone, etc. Obviously it's hard to say with what we know now. Humans have difficulty recognizing cross-ethnic faces, however I don't think dark skinned people are worse at recognizing same-ethnic faces than light skinned people. To me that suggests our social history is more to blame than some fundimental quality of skin tone.
mwcampbell 6y ago
But does the reason matter when the result is discrimination against people through no fault of their own?
- raxxorrax 6y ago
  Please discriminate against me too if that means facial recognition isn't working.
  - krisoft 6y ago
    That’s not necessarily what you want. “not working” can manifest itself in false positive matches too. Cue “all X people look the same” stupidness. And the truth is, the danger zone is when the algo is somewhat ok, but a few percents worse on the discriminated group. If it were obviously stupidly bad (like for example any two black people are a positive match according to it) then people will disregard it. But if its just slightly bad thats harder to notice and suddenly your loan/car rental is rejected because the security system says you look too much like a specific known fraudster. And of course they wont even tell you that thats the reason, because security.
    - raxxorrax 6y ago
      I worked in imaging for medical diagnostic support and dark complexions are just plainly more difficult to analyse because of restricted illumination times and limited contrast. Sensors got much better in recent years though. The indications in question were extremely rare for black people on the other hand. Another use case was the detection of Vitiligo, which is significantly more easy on black people. There was a great demand for therapies because patients are stigmatized for different reasons, especially in Africa because the symptoms can look similar to some deadly diseases. There is a lot of discrimination in imaging, but it doesn't have to do with race and more with the creation of binary images. As for facial recognition. I see some use cases for convenience, but it would be far more effective to disallow widespread application because of the numerous unsolved problems. So I see the move from IBM positively, even if they continue their work. In the end I believe face reg to be possible without bias, but I don't have these propped up security needs.
pfranz 6y ago
It goes even deeper than that. If you look at the history of photography, film, and television you'll see a bias towards white skin tones that standards have been built up around. Just looking at "leader ladies" or common reference images there isn't a bunch of diversity. In practice I've heard cinematographers and photographers bring it up around filming different ethnicities. It starts with changing the exposure, but means you have to spend more attention towards color reproduction. This can extend beyond the film or image sensor to biasing lighting equipment, defaults, standards, and thresholds. Looking through the literature you'll see things like, "the design decisions in the NTSC’s test images effectively biased the format toward rendering white people as more lifelike than other races" https://sterneworks.org/Mulvin-Sterne-Scenes.pdf https://sterneworks.org/Mulvin-Sterne-Scenes.pdf I'm mostly familiar with the history in the US. I'm earnestly curious about how other countries accommodated these types of things as they adopted the technologies.
- dnautics 6y ago
  It's deeper than that. The chemicals used in analog color photography didn't have a dye profile that could capture people of color effectively until the 80s.
  - sangnoir 6y ago
    Even deeper- Kodak only started putting effort to get the process right in the 70s after customers complained about poor color reproduction[1] in wood grain and chocolate 1. https://petapixel.com/2015/09/19/heres-a-look-at-how-color-film-was-originally-biased-toward-white-people/ https://petapixel.com/2015/09/19/heres-a-look-at-how-color-f...
  - solarkraft 6y ago
    I suspect it to be deeper than that: The relevant people didn't think it was worth it to invest all that much into developing it.
  - lallysingh 6y ago
    I'm guessing that nobody cared to try.
TuringNYC 6y ago
If there are technical challenges which would produced bias results...perhaps the answer is to not release the product. Perhaps do more R&D or more training on broader samples until one has a better product to release?
- xupybd 6y ago
  I don't think they are releasing products to make the world a better place. After all this tech is very dangerous. I don't the the ethics of racial bias are factored in to the decision making at these companies. This is one bias I think benefits folks with dark skin. I'd love to be in the group that doesn't get recognised by this terrifying tech.

robomartin 6y ago

> but the bias (and just sheer inaccuracy) I would hope that on HN one would take the time to explore the technical issues before even suggesting bias (which implies human racial discrimination of some sort). I am going to venture a guess that there's a large audience in the image processing/AI/ML world that lacks a fundamental understanding of image sensor and lens technology. I have never seen mention of concepts such as well capacity, quantum efficiency, noise floor, dynamic range, thermal noise, gamma encoding, compression induced errors, etc. in most work I have reviewed. Sensors used in the general class of imagers found in these experiments are nowhere near adequate to capture the full dynamics of a lot of real life images. The lowlights (referring to the lower portion of the dynamic range of a camera, encoding, compression and image processing system) can be some of the most challenging portions of the dynamic range to get quality data. The old idea applies: Garbage-in, garbage-out. It should come as no surprise that algorithms trained on (likely) bad images with bad lowlight detail will fail to deal with people of darker skin. It's almost a given. One can't assume cheap cameras and the data sets produced with these cameras will see the world the way our eyes are able to. Not to mention the fact that we have something called "understanding" while classifier systems have no clue whatsoever what they are looking at, all they can do is put things in buckets and that's that. In other words, there is no inherent comprehension of what a human being might be versus a bear or a teapot. That's a major problem. The answer isn't to give up. The answer is to understand and then go back and do it right. This isn't going to be cheap and it will likely require rethinking how we build and train these systems. As a tangentially related data point, I have three German Shepherd dogs. Two are the traditional black and brown coloring. The third is 100% black. It is virtually impossible to take a good picture of him. In anything but the right lighting he shows up as a dark amorphous blob. For all the prowess of the mighty camera in an iPhone 10, you'd be hard pressed to use those images to recognize him as anything other than a blob on a dark couch. I do have access to high performance images with far greater well capacity as well as 100% uncompressed data output. In that case there's usable data in the lowlights that, through gamma and LUT manipulation can be extracted. When you do that he quickly goes from looking like a blob to looking like a happy dog. Anyone interested in learning more, I would highly recommend looking up Jim Janesick: https://www.google.com/search?q=jim+janesick https://www.google.com/search?q=jim+janesick The popular phrase "he wrote the book" applies here. Jim's books on the subject of image sensor technology (science and design of sensors) are the reference work anyone in imaging studies. He designed so many sensors for space applications I am not sure he even remembers how many. I was fortunate enough to study CCD and CMOS sensor design under him a couple of decades ago. ML has to start with good data. Inadequate sensors coupled with compression and other processing artifacts leads to bad data, a formula for failure.

deleted 6y ago

[deleted]

electro_blah 6y ago

There's a simple fix. just hire a few white dudes to follow them around. this could lead to some parking related issues later though.

LanceH 6y ago
You can just hire white dudes. You'll need to hire some black dudes as well so there isn't a diversity gap.
legostormtroopr 6y ago
Yeah, but thats going to lead to a hiring bias. So half the people you hire should be black, but then you'll need white guys to follow them around. And for those who don't know the reference: https://www.imdb.com/title/tt1346402/ https://www.imdb.com/title/tt1346402/
- electro_blah 6y ago
  Yes, that's when the parking issue arise.

bilbo0s 6y ago

I've had this issue with AI's in the medical industry. At best, you have AI's that can easily recognize pathologies that an average rad could recognize, but are useless when it comes to trying to recognize a pathology that 99.999% of rads would miss. Why? Probably because the system is biased towards pathologies that rads recognize. Why? Probably because that's what's in the learning set. I understand all that, but that makes the AI almost useless in a production setting simply because of the way many healthcare delivery networks are structured with respect to radiology. At worst, you have AI's that seem to recognize pathologies that an average rad could recognize, but then inexplicably have a horrible miss on an obvious study that any first year radiologist would have caught while half asleep. It's the sort of miss that makes the astute observer wonder if the other vendor's AI, the one that didn't have any misses on your test set, was simply not fed an example study that triggered its blindspot? You start to wonder where the blindspot on that one was? You start to wonder does it even have a blindspot? How can you work around blindspots like this in general? Etc. But here's the thing, it's never good when you're thinking about mitigations while you're still testing the AI. My first RSNA was over 20 years ago. To this day, we're still hearing the same promises, and the production testing, (it never fails), is still uncovering the same issues. Now I would have thought that recognizing a human would be easier than trying to ascertain, say, calcification in a DX, but apparently these same issues crop up in a number of different applications of ML based technologies. I understand that a training set, by the very definition of the word "set", does not contain everything. Obviously, there will be bias towards whatever is in the training set. Essentially, most techniques today, simply train AI's to tell us whatever we tell the AI's to tell us. But for this reason it should be unsurprising that these sorts of AI's work best in settings with well defined domain spaces and are challenged when the domain space is less well defined. (And in either case, these AI's will have an inescapable bias towards whatever was in the training set.) What does a tumor look like? Is probably too broad a question to give these AI's. What does a stop sign look like? Is probably a question that these AI's could answer relatively reliably. (I hope?) You would have thought that "what does a human look like?" would be closer to "what does a stop sign look like?" But I guess it's not surprising to hear that it seems to trend towards "what does a tumor look like?" in practice.

jogundas 6y ago

jogundas 6y ago

From what I understand, NIST has been running a live "competition" on faces of 9 million people for some time already [0]. They are trying to evaluate racial biases in particular [1]. Was there a reason why you decided to not rely on what NIST was putting out? [0] https://www.nist.gov/programs-projects/face-recognition-vendor-test-frvt https://www.nist.gov/programs-projects/face-recognition-vend... [1] https://www.nist.gov/news-events/news/2019/12/nist-study-evaluates-effects-race-age-sex-face-recognition-software https://www.nist.gov/news-events/news/2019/12/nist-study-eva...

mschuster91 6y ago

> Put an African borrower in the frame and you got at best a failure to recognize a human, and at worst a reference to an animal. That happens uncomfortably often - and not even anywhere where AI is even remotely involved, like soap dispensers: https://reporter.rit.edu/tech/bigotry-encoded-racial-bias-technology https://reporter.rit.edu/tech/bigotry-encoded-racial-bias-te...

ipiz0618 6y ago

Improving the training set is the way to go. Like others have said, maybe we should attempt to improve contrast of darker faces before training (which won't be easy as well). That said, a non-diverse facial dataset in a diverse society like the US is simply useless. It doesn't help saying the AI is suffering from a human bias, and dropping these projects entirely, unless they are being used for a malign purpose like what China does.

userbinator 6y ago

A picture with a white college kid in the frame would get the output of "human". Put an African borrower in the frame and you got at best a failure to recognize a human, and at worst a reference to an animal. I wouldn't call that bias, but instead the pure and unadulterated innocent honesty of a machine doing what it does best: finding a closest match. A (white or otherwise non-African) human with zero knowledge of race and racism would probably give exactly the same result.