4 ms·

Thanks for sharing this article. Can someone knowledgeable about this issue explain why this is not a data issue? I have read people claiming that ML researcher

by malshe 5y ago

Thanks for sharing this article. Can someone knowledgeable about this issue explain why this is not a data issue? I have read people claiming that ML researchers may bring their own biases into the models but I haven't seen any concrete example of that. Even in the Twitter exchange in this article, Gebru doesn't explain how this is not just data bias. She just throws a lot of insults at LeCun but anyone can do that, right? I would have loved to see her explanation as she is the expert in this area.

visarga 5y ago
Well, technically, the way you choose the algorithm and set the hyper-parameters can influence accuracy in a non-uniform way over the distribution of data, introducing additional bias. Bias can be introduced at every step in the ML pipeline, from data collection to labeling, model selection, training, evaluation, deployment and monitoring. Not to mention the complex effects of feedback when you collect data that has been influenced by biases in a prior iteration of the model. That said, I am disappointed in Gebru, I read her "Gender Shades" paper where she completely ignored the existence of the Asian race. That's a pretty big limitation for a paper named in such general terms. In another place she labels Asians as "white adjacent" [1]. That doesn't seem to be ethical behavior for an ethics researcher. She also refused to debate with Yann LeCun on the topic of bias, instead sent him to reeducate himself [2]. [1] https://twitter.com/timnitgebru/status/1252752743942328321 https://twitter.com/timnitgebru/status/1252752743942328321 [2] https://twitter.com/timnitgebru/status/1275191515380215808 https://twitter.com/timnitgebru/status/1275191515380215808
- 908B64B197 5y ago
  > That said, I am disappointed in Gebru, I read her "Gender Shades" paper where she completely ignored the existence of the Asian race. That's a pretty big limitation for a paper named in such general terms. In another place she labels Asians as "white adjacent" You are scratching the surface of a much bigger racial divide in the Bay Area. [0] [1] > That doesn't seem to be ethical behavior for an ethics researcher. She also refused to debate with Yann LeCun on the topic of bias, instead sent him to reeducate himself I don't think she had the technical background required to really argue with LeCun. [0] https://www.nytimes.com/2010/05/02/us/02sfcrime.html https://www.nytimes.com/2010/05/02/us/02sfcrime.html [1] https://www.hup.harvard.edu/catalog.php?isbn=9780674018440 https://www.hup.harvard.edu/catalog.php?isbn=9780674018440
- malshe 5y ago
  Thanks for the reply. I’m sorry I still don’t see an example of the bias due to hyperparameter selection or tuning. Or model selection. The bias due to data is pretty clear to me and I see so many examples in the wild.
  - deleted 5y ago
    [deleted]
  - visarga 5y ago
    This is a great video about this exact issue. I couldn't have said it better: https://youtu.be/J7CrtblmMnU https://youtu.be/J7CrtblmMnU More concretely, say you train a gender detection model on people of all races. The way you do image preprocessing might influence results because black people faces have less contrast. The way you pick your regularisation could influence how well your model works on outliers (L1 reg tends to be less sensitive than L2). The way you set class weights could influence the trade-off between accuracy on different classes. Other algorithmic choices influence if the model does well on longer or shorter sequences (transformer vs LSTM). This might not seem socially biased, but if your classes correlate with input size, then the length bias might convert into class bias. I think math itself is socially unbiased (duh!), but models could amplify preexisting dataset biases when they correlate with some quality of the data such as its volume, resolution, contrast or complexity.
- random314 5y ago
  Gebru has been a terrible representative for AI ethics all around, but this was inevitable. "AI Ethics" is a pop science topic based on fantasies derived from the Terminator movie series.[1] When the foundation of the entire field is shaky it attracts careerist opportunists who have found an easy way to score scientific reputation points using shoddy or non-existent science. Google doubled down on this to build up a facade of an ethical company (guys, lets pretend project Dragonfly doesn't exist) and appointed the loudest person they could find in this pseudo science group. The inevitable happened - inevitable if you are familiar with Gebru's forced confrontations with LeCun. She publishes a bogus paper about the carbon footprint of training AI models, comparing it with jet exhaust. A paper more focused on rhetoric than substance. Completely ignoring that Google is carbon neutral and AI chips can run on solar power - just like a Tesla car. No reference is made to how these language models enable cross-language communication, eliminate driving around in circles in a foreign country because you don't speak the language and end up saving CO2. Google, by any measure, saves CO2 emissions for the user. Instead we have a dummy paper pretending that Google uses jet fuel for a one time training of an AI model. How low can "scientific research" fall. [1] Certainly, there needs to be work done to remove bias from public datasets; but there is not much to be done beyond this. NLP programs having great models for English and poor ones for Swahili is, frankly speaking, not a problem for ML algorithm researchers to solve. There are one-shot learning approaches but these are not motivated by the desire to eliminate bias.
  - visarga 5y ago
    I think Gebru is serving two masters. She wants societal change based on critical social justice and at the same time she publishes papers on AI ethics. A conflict of interests, you can't be an unbiased scientist while being a die hard activist at the same time. CJ ideology says to avoid debating with people of the "oppressing class" in order to not give them a platform, or because language itself is biased, or because the opponents are believed to act in bad faith, while the scientific method requires unbiased, impersonal, open debate. See the quote about "master's tools will never dismantle the master's house" for reference. They also practice cancelling their opponents instead of debating, which Gebru and her cohort tried to do on Twitter.

by malshe 5y ago

visarga 5y ago

Well, technically, the way you choose the algorithm and set the hyper-parameters can influence accuracy in a non-uniform way over the distribution of data, introducing additional bias. Bias can be introduced at every step in the ML pipeline, from data collection to labeling, model selection, training, evaluation, deployment and monitoring. Not to mention the complex effects of feedback when you collect data that has been influenced by biases in a prior iteration of the model. That said, I am disappointed in Gebru, I read her "Gender Shades" paper where she completely ignored the existence of the Asian race. That's a pretty big limitation for a paper named in such general terms. In another place she labels Asians as "white adjacent" [1]. That doesn't seem to be ethical behavior for an ethics researcher. She also refused to debate with Yann LeCun on the topic of bias, instead sent him to reeducate himself [2]. [1] https://twitter.com/timnitgebru/status/1252752743942328321 https://twitter.com/timnitgebru/status/1252752743942328321 [2] https://twitter.com/timnitgebru/status/1275191515380215808 https://twitter.com/timnitgebru/status/1275191515380215808

908B64B197 5y ago
> That said, I am disappointed in Gebru, I read her "Gender Shades" paper where she completely ignored the existence of the Asian race. That's a pretty big limitation for a paper named in such general terms. In another place she labels Asians as "white adjacent" You are scratching the surface of a much bigger racial divide in the Bay Area. [0] [1] > That doesn't seem to be ethical behavior for an ethics researcher. She also refused to debate with Yann LeCun on the topic of bias, instead sent him to reeducate himself I don't think she had the technical background required to really argue with LeCun. [0] https://www.nytimes.com/2010/05/02/us/02sfcrime.html https://www.nytimes.com/2010/05/02/us/02sfcrime.html [1] https://www.hup.harvard.edu/catalog.php?isbn=9780674018440 https://www.hup.harvard.edu/catalog.php?isbn=9780674018440
malshe 5y ago
Thanks for the reply. I’m sorry I still don’t see an example of the bias due to hyperparameter selection or tuning. Or model selection. The bias due to data is pretty clear to me and I see so many examples in the wild.
- deleted 5y ago
  [deleted]
- visarga 5y ago
  This is a great video about this exact issue. I couldn't have said it better: https://youtu.be/J7CrtblmMnU https://youtu.be/J7CrtblmMnU More concretely, say you train a gender detection model on people of all races. The way you do image preprocessing might influence results because black people faces have less contrast. The way you pick your regularisation could influence how well your model works on outliers (L1 reg tends to be less sensitive than L2). The way you set class weights could influence the trade-off between accuracy on different classes. Other algorithmic choices influence if the model does well on longer or shorter sequences (transformer vs LSTM). This might not seem socially biased, but if your classes correlate with input size, then the length bias might convert into class bias. I think math itself is socially unbiased (duh!), but models could amplify preexisting dataset biases when they correlate with some quality of the data such as its volume, resolution, contrast or complexity.
random314 5y ago
Gebru has been a terrible representative for AI ethics all around, but this was inevitable. "AI Ethics" is a pop science topic based on fantasies derived from the Terminator movie series.[1] When the foundation of the entire field is shaky it attracts careerist opportunists who have found an easy way to score scientific reputation points using shoddy or non-existent science. Google doubled down on this to build up a facade of an ethical company (guys, lets pretend project Dragonfly doesn't exist) and appointed the loudest person they could find in this pseudo science group. The inevitable happened - inevitable if you are familiar with Gebru's forced confrontations with LeCun. She publishes a bogus paper about the carbon footprint of training AI models, comparing it with jet exhaust. A paper more focused on rhetoric than substance. Completely ignoring that Google is carbon neutral and AI chips can run on solar power - just like a Tesla car. No reference is made to how these language models enable cross-language communication, eliminate driving around in circles in a foreign country because you don't speak the language and end up saving CO2. Google, by any measure, saves CO2 emissions for the user. Instead we have a dummy paper pretending that Google uses jet fuel for a one time training of an AI model. How low can "scientific research" fall. [1] Certainly, there needs to be work done to remove bias from public datasets; but there is not much to be done beyond this. NLP programs having great models for English and poor ones for Swahili is, frankly speaking, not a problem for ML algorithm researchers to solve. There are one-shot learning approaches but these are not motivated by the desire to eliminate bias.
- visarga 5y ago
  I think Gebru is serving two masters. She wants societal change based on critical social justice and at the same time she publishes papers on AI ethics. A conflict of interests, you can't be an unbiased scientist while being a die hard activist at the same time. CJ ideology says to avoid debating with people of the "oppressing class" in order to not give them a platform, or because language itself is biased, or because the opponents are believed to act in bad faith, while the scientific method requires unbiased, impersonal, open debate. See the quote about "master's tools will never dismantle the master's house" for reference. They also practice cancelling their opponents instead of debating, which Gebru and her cohort tried to do on Twitter.