10 ms·

Google is investigating the actions of another top AI ethicist

SpicyLemonZest 6y ago
> In this instance, yesterday our systems detected that an account had exfiltrated thousands of files and shared them with multiple external accounts. I mean, come on. There’s no way someone in a leadership position would legitimately think that’s okay. I suppose to some degree we should reserve judgment until hearing her side of the story - maybe this was just unambiguous evidence destined for her lawyer and the EEOC - but it’s hard to avoid some pretty negative conclusions about the prevailing culture in this group.
ALittleLight 6y ago
I was disappointed to find out that AI ethicists at Google aren't LessWrong style AI ethicists. I've just read one Google AI ethics paper in it's entirety but it seemed pretty non-insightful and not actionable. That said, if Google is "investigating" her because she was trying to find evidence of discrimination or bad treatment of Dr. Gebru, that seems borderline criminal behavior. (Assuming her attempts at trying to find evidence did not involve exfiltrating secret information).
- actuator 6y ago
  I think sending it to external accounts probably triggered this, there is no way this would not have raised a red flag in any company. I mean, Alphabet is the same company where a person took self driving documents with him to start a new company. Hopefully, if it was just normal conversations and after identities had been redacted, their access is restored. Still, analysing this information on their work machine itself and then sending out a summary seems like a better course of action than this.
  - fossuser 6y ago
    Somewhat related - Levandowski (the crook that stole the self driving docs) was just pardoned by Trump. The pardon was supported by Thiel, Luckey, and other Founders Fund people. I find this pretty irritating, but I guess when you’re in the company of actual war criminals like Eddie Gallagher then the bar is pretty low.[0] —— Separately, from the drama I’ve seen around the recent AI ethics stuff, I’d bet money Google is on the right side of this and the activists are not. [0]: https://news.ycombinator.com/item?id=25843040 https://news.ycombinator.com/item?id=25843040
    - bergstromm466 6y ago
      > Levandowski (the crook that stole the self driving docs) Why is your username 'foss user' when you're showing you are for intellectual property rights by calling this person a crook?
  - ALittleLight 6y ago
    True, but it's only Google/Alphabet saying she sent data to external accounts and I don't think they're especially trustworthy. If she sent secret info to outsiders she deserves to be fired, but if Google is investigating her because they think she's looking for evidence of discrimination then Google deserves whatever civil and legal penalties apply.
- cauthon 6y ago
  > I was disappointed to find out that AI ethicists at Google aren't LessWrong style AI ethicists. I've just read one Google AI ethics paper in it's entirety but it seemed pretty non-insightful and not actionable. This seems like a contradictory criticism to make. The Less Wrong folk, in my perception, mostly dabble in thought experiments (such as Roko’s basilisk) that aren’t terribly relevant to real world AI work. Timnit’s work on identifying racial and gendered inaccuracies in facial recognition [1] seems much more actionable. [1] https://www.technologyreview.com/2020/06/12/1003482/amazon-stopped-selling-police-face-recognition-fight/ https://www.technologyreview.com/2020/06/12/1003482/amazon-s...
  - ALittleLight 6y ago
    Gebru's (and co-author's) work on racial error in facial recognition was done while they were academics at Stanford and MIT respectively and not while a Google AI ethicist to my knowledge. Roko's Basilisk is a banned comment from Less Wrong (though interesting) and doesn't really represent the totality of that site or the associated ideas. https://news.mit.edu/2018/study-finds-gender-skin-type-bias-artificial-intelligence-systems-0212 https://news.mit.edu/2018/study-finds-gender-skin-type-bias-...
    - fossuser 6y ago
      Yeah - the basilisk thing is mostly a joke as far as I can tell and has little to do with the AGI goal alignment friendly AI work or the control problem. Miri is mostly focused on the AGI control problem and thinks that AGI is closer than others believe. If true, all other problems are pretty irrelevant. There is real work in ML bias and fairness to be done too, but it seems overrun by toxic personalities and partisan politics.
    - cauthon 6y ago
      I’m not sure which Google AI paper you’re referencing. The recent paper at the center of the controversy did have actionable suggestions regarding model training costs (financial and environmental), appropriate training sets, and generally incorporating domain expertise to avoid misinformation or mistranslations. [1] That’s fair, I haven’t visited LW in a while. I did thoroughly enjoy HPMOR, but thought that most of Eliezer’s other work fell closer to speculation than practice. As other commenters have mentioned, these AI algorithms are already being implemented in the real world with real consequences. Their associated ethical concerns therefore seem more urgent, and they can be acted upon. [1] https://www.technologyreview.com/2020/12/04/1013294/google-ai-ethics-research-paper-forced-out-timnit-gebru/ https://www.technologyreview.com/2020/12/04/1013294/google-a...
- unhammer 6y ago
  https://www.technologyreview.com/2020/12/04/1013294/google-ai-ethics-research-paper-forced-out-timnit-gebru/ https://www.technologyreview.com/2020/12/04/1013294/google-a... is a summary of the paper that started it, full of actionable points – and co-authored with Emily Bender, no less! Also, if you're into the LessWrong mindset and are susceptible to think that that which can be destroyed by the truth should be, you might find this tidbit interesting: > Buried in the recent trillion parameter language model paper is how the dataset to train it was created. Any page that contained one of these words was excluded: https://github.com/LDNOOBW/List-of-Dirty-Naughty-Obscene-and-Otherwise-Bad-Words/blob/master/en https://github.com/LDNOOBW/List-of-Dirty-Naughty-Obscene-and... Two sample banned words: "twink" and "sex" https://twitter.com/willie_agnew/status/1350551463718621184 https://twitter.com/willie_agnew/status/1350551463718621184
  - ALittleLight 6y ago
    Could you elaborate on this a little more? I'm not sure I'm connecting all the dots.
  - dgellow 6y ago
    I read your links and I have to say that I do not understand how the list of excluded words relates to the rest. And what’s the issue here.
firebaze 6y ago
Another perspective on this drama to me is what the official expectation of a team labelled "Ethical AI" is/was. I doubt the only focus was to prevent discrimination by sex or race, For example, there's also prevention of mass manipulation (AI learns human psychology, $BAD_ACTOR uses that to manipulates masses) and AI based profile generation of individuals with the goal of creating leverage for blackmail/extortion/whatever. Or, how would one form a controlling body of unethical usage in AI in such areas. There are quite a few aspects never mentioned in these discussions, and a search via google scholar turns up lots of the sex/gender/race papers, but none of the others (at least on the first few pages)-
- freeone3000 6y ago
  Those things are bad, and are worth thinking about, but right now we have more immediate concerns. AI systems are currently being used for bail eligibility, likelihood of recurrence, mortgage approval, and hiring. These are things that literally change people's lives on an individual level, and the systems being used are currently indecipherable black boxes that return a result with no possibility of meaningful appeal. There's a sense of starting small - redlining is illegal. No company wants to redline - it cuts into their profits and dings their compliance scores. So they're willing to work with us. Once we get that right, then, maybe we can start dealing with the cases where there aren't millions of dollars of funding assisting with making the tech more ethical -- and then, cases where it may actually be working against the money.
  - valesco 6y ago
    European law foresees those exact cases and IIRC prohibits automated treatment of personal information that leads to decisions that affects someone’s life, if no human takes the final decision and if the specific elements for the decision cannot be explained. In my opinion this forbids black boxes.
    - cauthon 6y ago
      I think you’re referring to GDPR article 22, which prohibits decisions based solely on automated processing but imposes no constraints on the transparency of such processing. [1] [1] https://gdpr-text.com/read/article-22/ https://gdpr-text.com/read/article-22/
KirillPanov 6y ago
Do Google corporate email accounts allow IMAP sync, like gmail accounts? I mean fully-replicated BODY.RFC822 sync, not just headers. I have to assume not, otherwise everybody using this feature would trigger the same exfiltration-lockout.
- KirillPanov 6y ago
  Or instead of answering my question people could just downvote me. I guess. Legitimately curious here. "Download all my mail from the mail server" is something I do multiple times every day. I guess my workflow would be verboten in corporate environments? Is it really such an odd behavior that they have an automated check for this, and it isn't throwing false positives every hour? Or is this one of those things they tell you at the "onboarding" like "hey, you might download all your personal gmail via IMAP regularly, but don't do that with your corporate email account or you'll get a phone call from security". Maybe I'm just out of touch with how corporate email works. It's been a decade since I was an employee of a publicly traded company.
  - willxinc 6y ago
    Typically this is disallowed from non company devices. If you're allowed to sync your corporate email to your personal devices, that's probably an oversight.
    - deleted 6y ago
      [deleted]
    - KirillPanov 6y ago
      So if IMAP-sync to corporate-owned laptops is a common and allowed use case how did an automated system notice this activity? It's not like the IMAP protocol sends the host MAC address or /etc/machine-id as part of the command stream. I'm really having a hard time imagining a plausible automated security system that would have led to this outcome.
      - 6y ago
esja 6y ago
“In this instance, yesterday our systems detected that an account had exfiltrated thousands of files and shared them with multiple external accounts.” If true, that is damning, and would demonstrate once again that being an “ethics researcher” does not mean that you are are any more ethical than the average person. It just means you are more interested in the subject. As a side note, I wish this field was more interested in meta-ethics than it is in forcing machines to abide by the personal ethics of the humans involved.
- nostromo 6y ago
  Many of the people in this field aren’t ethicists, they’re activists.
  - markdown 6y ago
    In a highly unethical environment, all ethicists are activists by necessity.
    - throwawaygulf 6y ago
      In reality, all activist ethicists are unethical by nature.
  - NotPavlovsDog 6y ago
    A large part of them believe that a researcher has to be an activist, i.e. research=activism. Many recent textbooks on social science see the following as credible resarch: "Emancipatory research: Research that exposes underlying ideologies in order to liberate those oppressed by them." (Zina O'Leary textbook, used by many schools, over 2K citations for multiple editions) So if a corporate behemoth such as Google wants to ethicswash itself by hiring individuals with the above approach, what outcome did it expect?
    - visarga 6y ago
      It was clearly a mistake. They don't act in a constructive way. Let's take Timnit's paper for example. She found out that Google had been using a biased language model in search. The bias was like - 'male' association with 'doctor' as opposed to 'woman' association to 'nurse'. But she didn't show how this is harming anyone in a concrete way. Just theoretical. And then she offered no solution, just blaming the work of others, using her paper as a soap box to raise scandal and make herself holier than thou.
ranjitcool1 6y ago
Just get rid of this meaningless ethical department. Have lawyers worry about ethics, Google focus on your business and continue to grow it. People think companies are democracies - just quit the job instead of feeling the need to change the company because that will just not happen and you will have unnecessary stress
villgax 6y ago
Lol, gogglers think company rules don't apply to them regarding company information?
throwawaygulf 6y ago
Now that Trump is out, and virtue signaling is passe, I hope we see Bay Area companies start an inquisition against SJWs, their cancerous "woke" ideology, and activist employees. The tide is slowly turning on them, for the betterment of society.
- approxim8ion 6y ago
  So you're calling for a purge, just so we're clear?
  - throwawaygulf 6y ago
    Excision.
    - approxim8ion 6y ago
      Thanks for the clarity.
      - throwawaygulf 6y ago
        You excise cancer.
knorker 6y ago
Are all ethicists as unethical as these two? I wonder if this one collected all those internal threads where Timnit was being toxic, that insiders kept describing. No? I guess this ethicist thought it best to only exfiltrate things matching her narrative?
- disposekinetics 6y ago
  Having worked in the industry for a long time it seems that anyone who considers themselves an activist believes that the ends justify their means.

> In this instance, yesterday our systems detected that an account had exfiltrated thousands of files and shared them with multiple external accounts. I mean, come on. There’s no way someone in a leadership position would legitimately think that’s okay. I suppose to some degree we should reserve judgment until hearing her side of the story - maybe this was just unambiguous evidence destined for her lawyer and the EEOC - but it’s hard to avoid some pretty negative conclusions about the prevailing culture in this group.

ALittleLight 6y ago

I was disappointed to find out that AI ethicists at Google aren't LessWrong style AI ethicists. I've just read one Google AI ethics paper in it's entirety but it seemed pretty non-insightful and not actionable. That said, if Google is "investigating" her because she was trying to find evidence of discrimination or bad treatment of Dr. Gebru, that seems borderline criminal behavior. (Assuming her attempts at trying to find evidence did not involve exfiltrating secret information).

actuator 6y ago
I think sending it to external accounts probably triggered this, there is no way this would not have raised a red flag in any company. I mean, Alphabet is the same company where a person took self driving documents with him to start a new company. Hopefully, if it was just normal conversations and after identities had been redacted, their access is restored. Still, analysing this information on their work machine itself and then sending out a summary seems like a better course of action than this.
- fossuser 6y ago
  Somewhat related - Levandowski (the crook that stole the self driving docs) was just pardoned by Trump. The pardon was supported by Thiel, Luckey, and other Founders Fund people. I find this pretty irritating, but I guess when you’re in the company of actual war criminals like Eddie Gallagher then the bar is pretty low.[0] —— Separately, from the drama I’ve seen around the recent AI ethics stuff, I’d bet money Google is on the right side of this and the activists are not. [0]: https://news.ycombinator.com/item?id=25843040 https://news.ycombinator.com/item?id=25843040
  - bergstromm466 6y ago
    > Levandowski (the crook that stole the self driving docs) Why is your username 'foss user' when you're showing you are for intellectual property rights by calling this person a crook?
- ALittleLight 6y ago
  True, but it's only Google/Alphabet saying she sent data to external accounts and I don't think they're especially trustworthy. If she sent secret info to outsiders she deserves to be fired, but if Google is investigating her because they think she's looking for evidence of discrimination then Google deserves whatever civil and legal penalties apply.
cauthon 6y ago
> I was disappointed to find out that AI ethicists at Google aren't LessWrong style AI ethicists. I've just read one Google AI ethics paper in it's entirety but it seemed pretty non-insightful and not actionable. This seems like a contradictory criticism to make. The Less Wrong folk, in my perception, mostly dabble in thought experiments (such as Roko’s basilisk) that aren’t terribly relevant to real world AI work. Timnit’s work on identifying racial and gendered inaccuracies in facial recognition [1] seems much more actionable. [1] https://www.technologyreview.com/2020/06/12/1003482/amazon-stopped-selling-police-face-recognition-fight/ https://www.technologyreview.com/2020/06/12/1003482/amazon-s...
- ALittleLight 6y ago
  Gebru's (and co-author's) work on racial error in facial recognition was done while they were academics at Stanford and MIT respectively and not while a Google AI ethicist to my knowledge. Roko's Basilisk is a banned comment from Less Wrong (though interesting) and doesn't really represent the totality of that site or the associated ideas. https://news.mit.edu/2018/study-finds-gender-skin-type-bias-artificial-intelligence-systems-0212 https://news.mit.edu/2018/study-finds-gender-skin-type-bias-...
  - fossuser 6y ago
    Yeah - the basilisk thing is mostly a joke as far as I can tell and has little to do with the AGI goal alignment friendly AI work or the control problem. Miri is mostly focused on the AGI control problem and thinks that AGI is closer than others believe. If true, all other problems are pretty irrelevant. There is real work in ML bias and fairness to be done too, but it seems overrun by toxic personalities and partisan politics.
  - cauthon 6y ago
    I’m not sure which Google AI paper you’re referencing. The recent paper at the center of the controversy did have actionable suggestions regarding model training costs (financial and environmental), appropriate training sets, and generally incorporating domain expertise to avoid misinformation or mistranslations. [1] That’s fair, I haven’t visited LW in a while. I did thoroughly enjoy HPMOR, but thought that most of Eliezer’s other work fell closer to speculation than practice. As other commenters have mentioned, these AI algorithms are already being implemented in the real world with real consequences. Their associated ethical concerns therefore seem more urgent, and they can be acted upon. [1] https://www.technologyreview.com/2020/12/04/1013294/google-ai-ethics-research-paper-forced-out-timnit-gebru/ https://www.technologyreview.com/2020/12/04/1013294/google-a...
unhammer 6y ago
https://www.technologyreview.com/2020/12/04/1013294/google-ai-ethics-research-paper-forced-out-timnit-gebru/ https://www.technologyreview.com/2020/12/04/1013294/google-a... is a summary of the paper that started it, full of actionable points – and co-authored with Emily Bender, no less! Also, if you're into the LessWrong mindset and are susceptible to think that that which can be destroyed by the truth should be, you might find this tidbit interesting: > Buried in the recent trillion parameter language model paper is how the dataset to train it was created. Any page that contained one of these words was excluded: https://github.com/LDNOOBW/List-of-Dirty-Naughty-Obscene-and-Otherwise-Bad-Words/blob/master/en https://github.com/LDNOOBW/List-of-Dirty-Naughty-Obscene-and... Two sample banned words: "twink" and "sex" https://twitter.com/willie_agnew/status/1350551463718621184 https://twitter.com/willie_agnew/status/1350551463718621184
- ALittleLight 6y ago
  Could you elaborate on this a little more? I'm not sure I'm connecting all the dots.
- dgellow 6y ago
  I read your links and I have to say that I do not understand how the list of excluded words relates to the rest. And what’s the issue here.

firebaze 6y ago

Another perspective on this drama to me is what the official expectation of a team labelled "Ethical AI" is/was. I doubt the only focus was to prevent discrimination by sex or race, For example, there's also prevention of mass manipulation (AI learns human psychology, $BAD_ACTOR uses that to manipulates masses) and AI based profile generation of individuals with the goal of creating leverage for blackmail/extortion/whatever. Or, how would one form a controlling body of unethical usage in AI in such areas. There are quite a few aspects never mentioned in these discussions, and a search via google scholar turns up lots of the sex/gender/race papers, but none of the others (at least on the first few pages)-

freeone3000 6y ago
Those things are bad, and are worth thinking about, but right now we have more immediate concerns. AI systems are currently being used for bail eligibility, likelihood of recurrence, mortgage approval, and hiring. These are things that literally change people's lives on an individual level, and the systems being used are currently indecipherable black boxes that return a result with no possibility of meaningful appeal. There's a sense of starting small - redlining is illegal. No company wants to redline - it cuts into their profits and dings their compliance scores. So they're willing to work with us. Once we get that right, then, maybe we can start dealing with the cases where there aren't millions of dollars of funding assisting with making the tech more ethical -- and then, cases where it may actually be working against the money.
- valesco 6y ago
  European law foresees those exact cases and IIRC prohibits automated treatment of personal information that leads to decisions that affects someone’s life, if no human takes the final decision and if the specific elements for the decision cannot be explained. In my opinion this forbids black boxes.
  - cauthon 6y ago
    I think you’re referring to GDPR article 22, which prohibits decisions based solely on automated processing but imposes no constraints on the transparency of such processing. [1] [1] https://gdpr-text.com/read/article-22/ https://gdpr-text.com/read/article-22/

KirillPanov 6y ago

Do Google corporate email accounts allow IMAP sync, like gmail accounts? I mean fully-replicated BODY.RFC822 sync, not just headers. I have to assume not, otherwise everybody using this feature would trigger the same exfiltration-lockout.

KirillPanov 6y ago
Or instead of answering my question people could just downvote me. I guess. Legitimately curious here. "Download all my mail from the mail server" is something I do multiple times every day. I guess my workflow would be verboten in corporate environments? Is it really such an odd behavior that they have an automated check for this, and it isn't throwing false positives every hour? Or is this one of those things they tell you at the "onboarding" like "hey, you might download all your personal gmail via IMAP regularly, but don't do that with your corporate email account or you'll get a phone call from security". Maybe I'm just out of touch with how corporate email works. It's been a decade since I was an employee of a publicly traded company.
- willxinc 6y ago
  Typically this is disallowed from non company devices. If you're allowed to sync your corporate email to your personal devices, that's probably an oversight.
  - deleted 6y ago
    [deleted]
  - KirillPanov 6y ago
    So if IMAP-sync to corporate-owned laptops is a common and allowed use case how did an automated system notice this activity? It's not like the IMAP protocol sends the host MAC address or /etc/machine-id as part of the command stream. I'm really having a hard time imagining a plausible automated security system that would have led to this outcome.
    - 6y ago

esja 6y ago

“In this instance, yesterday our systems detected that an account had exfiltrated thousands of files and shared them with multiple external accounts.” If true, that is damning, and would demonstrate once again that being an “ethics researcher” does not mean that you are are any more ethical than the average person. It just means you are more interested in the subject. As a side note, I wish this field was more interested in meta-ethics than it is in forcing machines to abide by the personal ethics of the humans involved.

nostromo 6y ago
Many of the people in this field aren’t ethicists, they’re activists.
- markdown 6y ago
  In a highly unethical environment, all ethicists are activists by necessity.
  - throwawaygulf 6y ago
    In reality, all activist ethicists are unethical by nature.
- NotPavlovsDog 6y ago
  A large part of them believe that a researcher has to be an activist, i.e. research=activism. Many recent textbooks on social science see the following as credible resarch: "Emancipatory research: Research that exposes underlying ideologies in order to liberate those oppressed by them." (Zina O'Leary textbook, used by many schools, over 2K citations for multiple editions) So if a corporate behemoth such as Google wants to ethicswash itself by hiring individuals with the above approach, what outcome did it expect?
  - visarga 6y ago
    It was clearly a mistake. They don't act in a constructive way. Let's take Timnit's paper for example. She found out that Google had been using a biased language model in search. The bias was like - 'male' association with 'doctor' as opposed to 'woman' association to 'nurse'. But she didn't show how this is harming anyone in a concrete way. Just theoretical. And then she offered no solution, just blaming the work of others, using her paper as a soap box to raise scandal and make herself holier than thou.

ranjitcool1 6y ago

Just get rid of this meaningless ethical department. Have lawyers worry about ethics, Google focus on your business and continue to grow it. People think companies are democracies - just quit the job instead of feeling the need to change the company because that will just not happen and you will have unnecessary stress

villgax 6y ago

Lol, gogglers think company rules don't apply to them regarding company information?

throwawaygulf 6y ago

Now that Trump is out, and virtue signaling is passe, I hope we see Bay Area companies start an inquisition against SJWs, their cancerous "woke" ideology, and activist employees. The tide is slowly turning on them, for the betterment of society.

approxim8ion 6y ago
So you're calling for a purge, just so we're clear?
- throwawaygulf 6y ago
  Excision.
  - approxim8ion 6y ago
    Thanks for the clarity.
    - throwawaygulf 6y ago
      You excise cancer.

knorker 6y ago

Are all ethicists as unethical as these two? I wonder if this one collected all those internal threads where Timnit was being toxic, that insiders kept describing. No? I guess this ethicist thought it best to only exfiltrate things matching her narrative?

disposekinetics 6y ago
Having worked in the industry for a long time it seems that anyone who considers themselves an activist believes that the ends justify their means.