5 ms·

So, unless I read the abstract wrong, this is identifying age, race, and gender, then making categorical qualifiers based on those distinctions. While using ML

So, unless I read the abstract wrong, this is identifying age, race, and gender, then making categorical qualifiers based on those distinctions. While using ML to do the facial recognition to distinguish a person's age, race and gender is neat - categorizing their political affiliation with a 72% accuracy rate is fairly nominal, given the tools used by modern parties to garner donations and directed online advertising - no offense.

tyingq 6y ago
"VGGFace224 was used to convert facial images into face descriptors, or 2,048-value-long vectors subsuming their core features." So they had more than age, race, and gender, but it doesn't really say how things were weighted.
- ad404b8a372f2b9 6y ago
  I don't really buy this study. If humans can't get a better accuracy than 55%, I'm convinced this vector is leaking some obvious (maybe high-frequency) information that doesn't have anything to do with the face. E.g location of the person.
  - tweetle_beetle 6y ago
    I hope you're right, otherwise we're in for some kind of AI phrenology nightmare in the not too distant future.
    - tomrod 6y ago
      We're pretty much there today.
      - mellavora 6y ago
        except for the phrenology part :)
        tomrod 6y ago
        I have built ML workflow for over a decade now. The amount of hogwash I've seen hocked definitely classifies as phrenology :)
  - gus_massa 6y ago
    The 55% is very fuzzy. They cite https://www.researchgate.net/profile/Konstantin-Tskhay/publication/232255935_Accuracy_in_Categorizing_Perceptually_Ambiguous_Groups_A_Review_and_Meta-Analysis/links/53f6b0220cf2fceacc74d1be/Accuracy-in-Categorizing-Perceptually-Ambiguous-Groups-A-Review-and-Meta-Analysis.pdf https://www.researchgate.net/profile/Konstantin-Tskhay/publi... The only relevant 55% that I can find is: > Allport and Kramer (1946) randomly presented 20 yearbook photographs of Jews and Non-Jews to 223 undergraduate students for 15 s each and asked them to categorize the person in each photograph as Jewish or non-Jewish, or to pass on the trial by indicating a lack of knowledge. The reported median identification for the sample was slightly above chance (55.5%; Allport & Kramer, 1946). Moreover, they found that highly prejudiced people were more accurate at distinguishing Jews from non-Jews So it's not the same set of photos and not the same question.
  - quasirandom 6y ago
    When I read a paper like this I'm looking for four things: (1) the data, (2) the benchmarks, (3) the architecture, (4) the controls/ablation. 1. The data: "We used a sample of 1,085,795 participants from three countries (the U.S., the UK, and Canada; see Table 1) and their self-reported political orientation, age, and gender. Their facial images (one per person) were obtained from their profiles on Facebook or a popular dating website... Facial images were processed using Face++37 to detect faces. Images were cropped around the face-box provided by Face++ (red frame on Fig. 1) and resized to 224 × 224 pixels." 2. The benchmarks: "For example, when asked to distinguish between two faces—one conservative and one liberal—people are correct about 55% of the time." 3. The controls: "What would an algorithm’s accuracy be when distinguishing between faces of people of the same age, gender, and ethnicity? To answer this question, classification accuracies were recomputed using only face pairs of the same age, gender, and ethnicity." A. A complaint: Geography and income are two powerful conditioners. These can leak in so many ways: uncropped background (geography), image color and quality (income), eyeglass shape (geography and income). This study really needs more controls. Geography and income would be a nice start.
    - sillysaurusx 6y ago
      Geography and income are two powerful conditioners. These can leak in so many ways: uncropped background (geography), image color and quality (income), eyeglass shape (geography and income). This study really needs more controls. Geography and income would be a nice start. But then the data wouldn't represent the natural world: nature as it is. Raw data is the correct thing to use, because it's what a hypothetical other person would also use if you ran the same experiment yourself.
      - deleted 6y ago
        [deleted]
      - SiempreViernes 6y ago
        Uh, the headline claim is about faces, how does it make sense to then insist that you must leave the background in?
  - im3w1l 6y ago
    It might pick up on scars, tattoos and piercings, testosterone level, diet (in particular weight), bags-under-eyes, glasses.
  - blowski 6y ago
    My political leanings change based on how recently a drank a cup of tea. This obsession with a left-right spectrum is a big part of the problem now. You think teachers are underpaid? Oh obviously you must be a pro-abortion, $15 minimum wage supporting, transgender-rights activist. What's that you say, Christian bakers should be allowed to refuse to bake a cake with a pro-gay message on it? Oh, you must be a gun-toting, pro-life, anti-immigratnt Trump fanatic. This kind of sorting people into simple binary categories, and giving them a "shopping bag" full of opinions they're supposed to hold helps nobody. I'm not sure how this was relevant in anyway to your comment, but I just kinda jumped on my soapbox there.
  - notahacker 6y ago
    Apart from the 55% human accuracy, which apparently comes from a completely unrelated study, the bit that really stands out to me is it reports the accuracy only drops from 72% to 68% when controlling for demographic accuracy in the US (a little more noticeable in the UK). Considering demographics alone gets you 60-90% accuracy on voting intention for many US demographics, it strikes me as extremely odd it would have have so little impact on the model.
- harias 6y ago
  >So they had more than age, race, and gender, Doesn't have to be that way. It could be age, age+1, age+2 ....
- theonlyklas 6y ago
  I believe that these descriptors are created only based off the visual image: A face descriptor is obtained from the learned networks as follows: the centre 224 × 224 crop of the face image is used. The shorter side is resized to 256, and the CNNs descriptor is computed for this region by extracting the deep features from the layer adjacent to the classifier layer. This leads to a 2048 dimensional descriptor, which is then L2 normalised. https://www.arxiv-vanity.com/papers/1710.08092/ https://www.arxiv-vanity.com/papers/1710.08092/
hirundo 6y ago
They tested this question specifically: > Both in real life and in our sample, the classification of political orientation is to some extent enabled by demographic traits clearly displayed on participants’ faces. For example ... white people, older people, and males are more likely to be conservatives. What would an algorithm’s accuracy be when distinguishing between faces of people of the same age, gender, and ethnicity? To answer this question, classification accuracies were recomputed using only face pairs of the same age, gender, and ethnicity ... The accuracy dropped by only 3.5% on average
- 11thEarlOfMar 6y ago
  From there, it would seem that cues might come from how 'kempt' they appear, whether the head shot was from a party or for a resume, perhaps color and style of clothing, ... I.e., maybe not strictly the face.
  - evan_ 6y ago
    If they’re wearing a face mask, if they’re carrying a tiki torch or storming the capitol building...
  - tyingq 6y ago
    "To minimize the role of the background and non-facial features, images were tightly cropped around the face" Though cropping can only do so much.
  - deleted 6y ago
    [deleted]
- kevinventullo 6y ago
  If the total population sampling is the same, I would expect the accuracies to remain the same. E.g. if I can get 72% accuracy in the total population just by looking at age/race/gender, doesn’t that exactly mean the accuracies in each individual category are on average 72%?
  - 6y ago
deleted 6y ago
[deleted]
karaterobot 6y ago
Does anyone know what the accuracy of a prediction is if you use only those three factors -- age, race, and gender?
- boomboomsubban 6y ago
  You can basically determine this by looking at voter/exit poll results, and any single criteria would give ~55% at best.
- gamegoblin 6y ago
  All numbers are Biden-Trump in 2020: People under 30: 60-36. White men: 38-61 Black women: 90-9 So there are definitely some strong predictors there. Source: https://www.businessinsider.com/2016-2020-electoral-maps-exit-polls-compared-2020-11 https://www.businessinsider.com/2016-2020-electoral-maps-exi...
  - evgen 6y ago
    If only the researchers were as smart as you apparently think you are and somehow remembered to control for these very obvious factors. Oh yeah, they did.
    - gamegoblin 6y ago
      I was not commenting on the main article about the research, but instead on the comment by user karaterobot that I am responding to which -- if I am interpreting it correctly -- asks the question how good one can guess purely based on those 3 demographic axes. As you note, the researcher's predictor appears to do better than random even controlling for these obvious demographic skews, which is fascinating.
      - stkdump 6y ago
        Actually, you have to answer something more from the other side: for a randomly picked person (voter), i.e. considering the distribution of race, gender and age in the population: how likely is a guess just based on these factors correct. The number you come up with might actually not be as far from 50% as you would expect.

tyingq 6y ago

"VGGFace224 was used to convert facial images into face descriptors, or 2,048-value-long vectors subsuming their core features." So they had more than age, race, and gender, but it doesn't really say how things were weighted.

ad404b8a372f2b9 6y ago
I don't really buy this study. If humans can't get a better accuracy than 55%, I'm convinced this vector is leaking some obvious (maybe high-frequency) information that doesn't have anything to do with the face. E.g location of the person.
- tweetle_beetle 6y ago
  I hope you're right, otherwise we're in for some kind of AI phrenology nightmare in the not too distant future.
  - tomrod 6y ago
    We're pretty much there today.
    - mellavora 6y ago
      except for the phrenology part :)
      - tomrod 6y ago
        I have built ML workflow for over a decade now. The amount of hogwash I've seen hocked definitely classifies as phrenology :)
- gus_massa 6y ago
  The 55% is very fuzzy. They cite https://www.researchgate.net/profile/Konstantin-Tskhay/publication/232255935_Accuracy_in_Categorizing_Perceptually_Ambiguous_Groups_A_Review_and_Meta-Analysis/links/53f6b0220cf2fceacc74d1be/Accuracy-in-Categorizing-Perceptually-Ambiguous-Groups-A-Review-and-Meta-Analysis.pdf https://www.researchgate.net/profile/Konstantin-Tskhay/publi... The only relevant 55% that I can find is: > Allport and Kramer (1946) randomly presented 20 yearbook photographs of Jews and Non-Jews to 223 undergraduate students for 15 s each and asked them to categorize the person in each photograph as Jewish or non-Jewish, or to pass on the trial by indicating a lack of knowledge. The reported median identification for the sample was slightly above chance (55.5%; Allport & Kramer, 1946). Moreover, they found that highly prejudiced people were more accurate at distinguishing Jews from non-Jews So it's not the same set of photos and not the same question.
- quasirandom 6y ago
  When I read a paper like this I'm looking for four things: (1) the data, (2) the benchmarks, (3) the architecture, (4) the controls/ablation. 1. The data: "We used a sample of 1,085,795 participants from three countries (the U.S., the UK, and Canada; see Table 1) and their self-reported political orientation, age, and gender. Their facial images (one per person) were obtained from their profiles on Facebook or a popular dating website... Facial images were processed using Face++37 to detect faces. Images were cropped around the face-box provided by Face++ (red frame on Fig. 1) and resized to 224 × 224 pixels." 2. The benchmarks: "For example, when asked to distinguish between two faces—one conservative and one liberal—people are correct about 55% of the time." 3. The controls: "What would an algorithm’s accuracy be when distinguishing between faces of people of the same age, gender, and ethnicity? To answer this question, classification accuracies were recomputed using only face pairs of the same age, gender, and ethnicity." A. A complaint: Geography and income are two powerful conditioners. These can leak in so many ways: uncropped background (geography), image color and quality (income), eyeglass shape (geography and income). This study really needs more controls. Geography and income would be a nice start.
  - sillysaurusx 6y ago
    Geography and income are two powerful conditioners. These can leak in so many ways: uncropped background (geography), image color and quality (income), eyeglass shape (geography and income). This study really needs more controls. Geography and income would be a nice start. But then the data wouldn't represent the natural world: nature as it is. Raw data is the correct thing to use, because it's what a hypothetical other person would also use if you ran the same experiment yourself.
    - deleted 6y ago
      [deleted]
    - SiempreViernes 6y ago
      Uh, the headline claim is about faces, how does it make sense to then insist that you must leave the background in?
- im3w1l 6y ago
  It might pick up on scars, tattoos and piercings, testosterone level, diet (in particular weight), bags-under-eyes, glasses.
- blowski 6y ago
  My political leanings change based on how recently a drank a cup of tea. This obsession with a left-right spectrum is a big part of the problem now. You think teachers are underpaid? Oh obviously you must be a pro-abortion, $15 minimum wage supporting, transgender-rights activist. What's that you say, Christian bakers should be allowed to refuse to bake a cake with a pro-gay message on it? Oh, you must be a gun-toting, pro-life, anti-immigratnt Trump fanatic. This kind of sorting people into simple binary categories, and giving them a "shopping bag" full of opinions they're supposed to hold helps nobody. I'm not sure how this was relevant in anyway to your comment, but I just kinda jumped on my soapbox there.
- notahacker 6y ago
  Apart from the 55% human accuracy, which apparently comes from a completely unrelated study, the bit that really stands out to me is it reports the accuracy only drops from 72% to 68% when controlling for demographic accuracy in the US (a little more noticeable in the UK). Considering demographics alone gets you 60-90% accuracy on voting intention for many US demographics, it strikes me as extremely odd it would have have so little impact on the model.
harias 6y ago
>So they had more than age, race, and gender, Doesn't have to be that way. It could be age, age+1, age+2 ....
theonlyklas 6y ago
I believe that these descriptors are created only based off the visual image: A face descriptor is obtained from the learned networks as follows: the centre 224 × 224 crop of the face image is used. The shorter side is resized to 256, and the CNNs descriptor is computed for this region by extracting the deep features from the layer adjacent to the classifier layer. This leads to a 2048 dimensional descriptor, which is then L2 normalised. https://www.arxiv-vanity.com/papers/1710.08092/ https://www.arxiv-vanity.com/papers/1710.08092/

hirundo 6y ago

They tested this question specifically: > Both in real life and in our sample, the classification of political orientation is to some extent enabled by demographic traits clearly displayed on participants’ faces. For example ... white people, older people, and males are more likely to be conservatives. What would an algorithm’s accuracy be when distinguishing between faces of people of the same age, gender, and ethnicity? To answer this question, classification accuracies were recomputed using only face pairs of the same age, gender, and ethnicity ... The accuracy dropped by only 3.5% on average

11thEarlOfMar 6y ago
From there, it would seem that cues might come from how 'kempt' they appear, whether the head shot was from a party or for a resume, perhaps color and style of clothing, ... I.e., maybe not strictly the face.
- evan_ 6y ago
  If they’re wearing a face mask, if they’re carrying a tiki torch or storming the capitol building...
- tyingq 6y ago
  "To minimize the role of the background and non-facial features, images were tightly cropped around the face" Though cropping can only do so much.
- deleted 6y ago
  [deleted]
kevinventullo 6y ago
If the total population sampling is the same, I would expect the accuracies to remain the same. E.g. if I can get 72% accuracy in the total population just by looking at age/race/gender, doesn’t that exactly mean the accuracies in each individual category are on average 72%?
- 6y ago

deleted 6y ago

[deleted]

karaterobot 6y ago

Does anyone know what the accuracy of a prediction is if you use only those three factors -- age, race, and gender?

boomboomsubban 6y ago
You can basically determine this by looking at voter/exit poll results, and any single criteria would give ~55% at best.
gamegoblin 6y ago
All numbers are Biden-Trump in 2020: People under 30: 60-36. White men: 38-61 Black women: 90-9 So there are definitely some strong predictors there. Source: https://www.businessinsider.com/2016-2020-electoral-maps-exit-polls-compared-2020-11 https://www.businessinsider.com/2016-2020-electoral-maps-exi...
- evgen 6y ago
  If only the researchers were as smart as you apparently think you are and somehow remembered to control for these very obvious factors. Oh yeah, they did.
  - gamegoblin 6y ago
    I was not commenting on the main article about the research, but instead on the comment by user karaterobot that I am responding to which -- if I am interpreting it correctly -- asks the question how good one can guess purely based on those 3 demographic axes. As you note, the researcher's predictor appears to do better than random even controlling for these obvious demographic skews, which is fascinating.
    - stkdump 6y ago
      Actually, you have to answer something more from the other side: for a randomly picked person (voter), i.e. considering the distribution of race, gender and age in the population: how likely is a guess just based on these factors correct. The number you come up with might actually not be as far from 50% as you would expect.