5 ms·

A large part of them believe that a researcher has to be an activist, i.e. research=activism. Many recent textbooks on social science see the following as credi

A large part of them believe that a researcher has to be an activist, i.e. research=activism. Many recent textbooks on social science see the following as credible resarch: "Emancipatory research: Research that exposes underlying ideologies in order to liberate those oppressed by them." (Zina O'Leary textbook, used by many schools, over 2K citations for multiple editions) So if a corporate behemoth such as Google wants to ethicswash itself by hiring individuals with the above approach, what outcome did it expect?

visarga 6y ago
It was clearly a mistake. They don't act in a constructive way. Let's take Timnit's paper for example. She found out that Google had been using a biased language model in search. The bias was like - 'male' association with 'doctor' as opposed to 'woman' association to 'nurse'. But she didn't show how this is harming anyone in a concrete way. Just theoretical. And then she offered no solution, just blaming the work of others, using her paper as a soap box to raise scandal and make herself holier than thou.
- NotPavlovsDog 6y ago
  With the hard realism approach, a corporate entity would fund an independent body of research, and then re-use publications that match their agenda. Hiring someone (perhaps with aspirations to buy their loyalty?) entails risks. Instead of pulling funding from the "independent think-tank", google now finds itself in the midst of a potential discrimination/political scandal involving an employee.
- thishit 6y ago
  Just shows that you didn't understand their paper then, if that's your disingenuous take on it.
  - visarga 6y ago
    Can you please enlighten me about her best constructive idea from this paper? No critique, just one original idea.
- gillesjacobs 6y ago
  While I am not a fan of Gebru's handling of her own firing, this is a mischaracterization of her work. She routinely offered suggested methods, experiments, and even new datasets [1] that fixed what she saw as wrong. She does have a fair bit of practical ML experience as shown in the computer vision papers in her publication list. 1. Buolamwini, Joy, and Timnit Gebru. "Gender shades: Intersectional accuracy disparities in commercial gender classification." Conference on fairness, accountability and transparency. 2018.
  - visarga 6y ago
    Of course I was aware of her Gender Shades paper. It's a small bias evaluation benchmark dataset. > We developed the Pilot Parliaments Benchmark (PPB) to achieve better intersectional representation on the basis of genderand skin type. PPB consists of 1270 individuals from three African countries (Rwanda, Senegal, South Africa) and three European countries (Ice-land, Finland, Sweden) selected for gender parity in the national parliaments. A dataset of 1270 images is hardly a breakthrough, the kind I expect to see in a small university project. But it doesn't lead to better models because it's not nearly large enough to train on. What it can do is rate existing models. Basically - useful to critique, not to improve. A small nitpick: why just two races in a de-biasing dataset? Where are the Asians?
    - njanirudh 6y ago
      ethic "saviors" don't consider asians as minority
- jsinai 6y ago
  > But she didn’t show how this is harming anyone in a concrete way. To roll with the example “man” ~ “doctor”, “woman” ~ “nurse”, the harm is having a giant and widely used search engine reinforce baseless gender biases, ie that there is no underlying reason why women should be nurses and men doctors. What is the harm you may ask? The harm may be subtle, eg being surprised when you find out your next doctor is a woman or your next is a man. It could suppress career choices and aspirations, and it could even be financial, eg reinforcing systemic pay gaps.
  - visarga 6y ago
    This kind of hypothetical effects should be documented in concrete cases by a good ethical scientist instead of just described from imagination. For example, when someone searches for a doctor, let's say a male comes up first - who stops at the first Google result? They would probably need to go deep and read about the doctor's experience and find patient reviews.
    - jsinai 6y ago
      Part of the issue is that the harms of gender bias (and other types of bias) should not need to be made explicit, but part of the research canon. Should a security researcher outline the harms of an attacker obtaining user credentials, or is our imagination sufficient because the harms are well known to us? And if you were looking for more in depth studies, then there is a ton of published research, maybe not all of it on arxiv or in machine learning journals.
      - visarga 6y ago
        At some point imagination has to make touch with reality otherwise it can become unhinged. Yes, security researchers can enumerate concrete cases where "the harms of an attacker obtaining user credentials" caused damage.
  - thu2111 6y ago
    That particular "bias" isn't actually a bias, it's an accurate learning about the distribution of genders between jobs in the real world. Very few professions have an exactly 50:50 balance of men and women. Most are tilted towards one gender or the other. The purpose of a correct search engine is not to reduce my "surprise" at arbitrary events but rather to give me the information I'm looking for, which will more often than not be questions about the real world - not the fever dream of some hard-left activist.
- hnlurker 6y ago
  I have no ML background, I’m not trying to be overly critical, but why would the association go in that direction? Wouldn’t the model be the other way? Ie doctor is male and nurse female. Also, it’s a pretty shitty model since females dominate the medical field in both real and per capita numbers.
n4r9 6y ago
I feel like your first two sentences are conflating the belief that research has to involve activism, and the belief that activist pursuits are worthy research. Why is the aim of exposing ideology not worthy of being researched in your opinion?
- NotPavlovsDog 6y ago
  Some concerns for social science involve the lack of replicability, publications thus becoming literature, with "credibility" often established by in-clique circular citations. Emotional and political coloring also come to mind. There is a position in social science that all research is political (Frankfurt school and those that oppose it, among others). The belief that research has to involve activism has been demonstrated by the individuals under discussion. I cite that this is taught as a valid position in the field. Political motivation of scientific activity is worthy of research. Both sides involved in the conflict at google may be described as demonstrating political, and apparently opposing, viewpoints.
throwawaygulf 6y ago
>A large part of them believe that a researcher has to be an activist, i.e. research=activism. A large part of them believe in other delusions such as "silence is violence", coming to work on time is "Whiteness", and "objective, rational" thinking is normalized racism. In their deranged minds, the ends justify their means, so they can be the antithesis of their being because their crusade is holy, and just. That's the complete opposite of being a researcher.

visarga 6y ago

It was clearly a mistake. They don't act in a constructive way. Let's take Timnit's paper for example. She found out that Google had been using a biased language model in search. The bias was like - 'male' association with 'doctor' as opposed to 'woman' association to 'nurse'. But she didn't show how this is harming anyone in a concrete way. Just theoretical. And then she offered no solution, just blaming the work of others, using her paper as a soap box to raise scandal and make herself holier than thou.

NotPavlovsDog 6y ago
With the hard realism approach, a corporate entity would fund an independent body of research, and then re-use publications that match their agenda. Hiring someone (perhaps with aspirations to buy their loyalty?) entails risks. Instead of pulling funding from the "independent think-tank", google now finds itself in the midst of a potential discrimination/political scandal involving an employee.
thishit 6y ago
Just shows that you didn't understand their paper then, if that's your disingenuous take on it.
- visarga 6y ago
  Can you please enlighten me about her best constructive idea from this paper? No critique, just one original idea.
gillesjacobs 6y ago
While I am not a fan of Gebru's handling of her own firing, this is a mischaracterization of her work. She routinely offered suggested methods, experiments, and even new datasets [1] that fixed what she saw as wrong. She does have a fair bit of practical ML experience as shown in the computer vision papers in her publication list. 1. Buolamwini, Joy, and Timnit Gebru. "Gender shades: Intersectional accuracy disparities in commercial gender classification." Conference on fairness, accountability and transparency. 2018.
- visarga 6y ago
  Of course I was aware of her Gender Shades paper. It's a small bias evaluation benchmark dataset. > We developed the Pilot Parliaments Benchmark (PPB) to achieve better intersectional representation on the basis of genderand skin type. PPB consists of 1270 individuals from three African countries (Rwanda, Senegal, South Africa) and three European countries (Ice-land, Finland, Sweden) selected for gender parity in the national parliaments. A dataset of 1270 images is hardly a breakthrough, the kind I expect to see in a small university project. But it doesn't lead to better models because it's not nearly large enough to train on. What it can do is rate existing models. Basically - useful to critique, not to improve. A small nitpick: why just two races in a de-biasing dataset? Where are the Asians?
  - njanirudh 6y ago
    ethic "saviors" don't consider asians as minority
jsinai 6y ago
> But she didn’t show how this is harming anyone in a concrete way. To roll with the example “man” ~ “doctor”, “woman” ~ “nurse”, the harm is having a giant and widely used search engine reinforce baseless gender biases, ie that there is no underlying reason why women should be nurses and men doctors. What is the harm you may ask? The harm may be subtle, eg being surprised when you find out your next doctor is a woman or your next is a man. It could suppress career choices and aspirations, and it could even be financial, eg reinforcing systemic pay gaps.
- visarga 6y ago
  This kind of hypothetical effects should be documented in concrete cases by a good ethical scientist instead of just described from imagination. For example, when someone searches for a doctor, let's say a male comes up first - who stops at the first Google result? They would probably need to go deep and read about the doctor's experience and find patient reviews.
  - jsinai 6y ago
    Part of the issue is that the harms of gender bias (and other types of bias) should not need to be made explicit, but part of the research canon. Should a security researcher outline the harms of an attacker obtaining user credentials, or is our imagination sufficient because the harms are well known to us? And if you were looking for more in depth studies, then there is a ton of published research, maybe not all of it on arxiv or in machine learning journals.
    - visarga 6y ago
      At some point imagination has to make touch with reality otherwise it can become unhinged. Yes, security researchers can enumerate concrete cases where "the harms of an attacker obtaining user credentials" caused damage.
- thu2111 6y ago
  That particular "bias" isn't actually a bias, it's an accurate learning about the distribution of genders between jobs in the real world. Very few professions have an exactly 50:50 balance of men and women. Most are tilted towards one gender or the other. The purpose of a correct search engine is not to reduce my "surprise" at arbitrary events but rather to give me the information I'm looking for, which will more often than not be questions about the real world - not the fever dream of some hard-left activist.
hnlurker 6y ago
I have no ML background, I’m not trying to be overly critical, but why would the association go in that direction? Wouldn’t the model be the other way? Ie doctor is male and nurse female. Also, it’s a pretty shitty model since females dominate the medical field in both real and per capita numbers.

n4r9 6y ago

I feel like your first two sentences are conflating the belief that research has to involve activism, and the belief that activist pursuits are worthy research. Why is the aim of exposing ideology not worthy of being researched in your opinion?

NotPavlovsDog 6y ago
Some concerns for social science involve the lack of replicability, publications thus becoming literature, with "credibility" often established by in-clique circular citations. Emotional and political coloring also come to mind. There is a position in social science that all research is political (Frankfurt school and those that oppose it, among others). The belief that research has to involve activism has been demonstrated by the individuals under discussion. I cite that this is taught as a valid position in the field. Political motivation of scientific activity is worthy of research. Both sides involved in the conflict at google may be described as demonstrating political, and apparently opposing, viewpoints.

throwawaygulf 6y ago

>A large part of them believe that a researcher has to be an activist, i.e. research=activism. A large part of them believe in other delusions such as "silence is violence", coming to work on time is "Whiteness", and "objective, rational" thinking is normalized racism. In their deranged minds, the ends justify their means, so they can be the antithesis of their being because their crusade is holy, and just. That's the complete opposite of being a researcher.