13 ms·

Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. They couch these in ter

by acoard 6y ago

Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. They couch these in terms of rigour, but the substance of the problem has to do with her criticisms against AI. Ultimately it makes the whole Ethical AI department look more like a rubber stamp for Google.

orsenthil 6y ago
> Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. I didn't read that. I read the person _demanded_ who said a particular critical feedback, or questioned the approaches instead of addressing them. The person gave the ultimatum to resign if details were not shared.
- sudosysgen 6y ago
  Unless she lied in her first e-mail, which it doesn't seem like she did, the reason she made those demands is because they asked for a retraction of the paper without indicating why the paper should be retracted. Asking for the identity of people that have the authority to ask for a withdrawal of your research without stating their issues with it seems understandable, if excessive. But maybe I misunderstood something.
  - lokar 6y ago
    AIUI, they asked her to retract it because she submitted it before getting final approval, and then they in fact decided not to approve it.
    - Footkerchief 6y ago
      Dean's statement is clear that it was approved before being submitted: > Unfortunately, this particular paper was only shared with a day’s notice before its deadline — we require two weeks for this sort of review — and then instead of awaiting reviewer feedback, it was approved for submission and submitted. A cross functional team then reviewed the paper as part of our regular process and the authors were informed that it didn’t meet our bar for publication and were given feedback about why. [...] We acknowledge that the authors were extremely disappointed with the decision that Megan and I ultimately made, especially as they’d already submitted the paper. There is no statement at all of how to reconcile "approved for submission" with "didn’t meet our bar for publication", which probably means that there is no reconciliation, and the cancellation was done outside normal process.
      - lokar 6y ago
        I see what you mean. I wonder if he is trying to say that there was a process error, it was approved without review (in error), she sent it out, and then they came back to her and said "wait, no, you can't publish that after all"
      - dash2 6y ago
        Sounds like the “editor” or analogous person didn’t wait to hear back from the “referees”. Whose fault this is is not made clear.
- UncleMeat 6y ago
  If your work is suddenly and unexpected roadblocked at the last second by internal review, the only way to make changes that prevent that from happening in the future is to clearly understand the situations and criticisms that led to the roadblock. This is why understanding who raised these concerns is important. Anonymous feedback blowing up a project at the final moments is sure to frustrate anybody. If what she has said is true then it was also very difficult for her to even have access to the substance of the critique in the first place, with the initial story from her management being that she would not be able to see the documents explaining why the paper was to be retracted. The critique here appears to have been fairly minor, too. Failing to cite some recent research is rarely grounds for rejection.
  - siliconmountain 6y ago
    Isn't the academic journal peer review process generally an anonymous feedback mechanism? Why does this need to be different?
    - DonaldPShimoda 6y ago
      Because this isn't peer review — or at least, it's not meant to be (per the top-level comment). That's the whole issue, really: there already exists a peer review process to ensure the paper's academic rigor, so why is Google hiding behind a claim of the necessity of anonymity for a corporate (not academic) process?
    - ethbr0 6y ago
      Because typically your academic journal peers don't work for the same bosses you do.
    - whimsicalism 6y ago
      > academic journal peer review process Fro my understanding, this paper had already passed peer review and been accepted. Google management then decided to block the publication using the IP review process.
      - 6y ago
- jonathankoren 6y ago
  Given that internal prepublication review at every company I've ever been with is merely there to avoid IP leakage, I find it very hard to believe that the feedback is is given in good faith. It's like the oil industry claiming that a climate change paper isn't talking enough about the economic benefits of growing citrus in Alaska. Quite frankly, there's simply no reason to address them, because the problems with BERT, exist with every LLM. Google stepped in and changed the procedure for this paper, because they wanted to spike it because they were embarrassed by it.
btown 6y ago
Let's be even more clear - they pushed out someone in their Ethical AI department because she wanted to have human conversations to determine the basis for being asked to soften critiques. It's one thing for reviewers, even anonymous reviewers, to reject a paper on its merits; it's another, in Timnit's own words [0], to be told "'it has been decided'" through "a privileged and confidential document to HR" despite clearing the subject matter beforehand. In light of a more general frustration, it's very reasonable for Timnit to escalate the situation by putting her own career on the table, simply to request that people engage with the paper rather than flat-out rejecting it. And if Jeff wants to respond by immediately cutting ties, and by putting out a document that doesn't even address the situation at hand (edit: much less the underlying issues of unequal treatment for women that Timnit describes)... that's a reflection of his ethics and the ethics of the company that stands behind him. [0] For those who haven't read Timnit's memo that Jeff references in the OP, it's worth reading: https://www.platformer.news/p/the-withering-email-that-got-an-ethical https://www.platformer.news/p/the-withering-email-that-got-a... EDIT 2: follow https://twitter.com/timnitGebru https://twitter.com/timnitGebru to see more of her side of the story. She retweeted https://www.wired.com/story/prominent-ai-ethics-researcher-says-google-fired-her/ https://www.wired.com/story/prominent-ai-ethics-researcher-s... as a good explanation of the situation for laypeople.
- tfehring 6y ago
  Also from Gebru's memo ([0] in the parent comment): >And you are told after a while, that your manager can read you a privileged and confidential document Emphasis mine. Showing your employee that you don't even trust her with a written copy of the rejection of her paper is not a great way to engender a good working relationship. Note that this pretty clearly seems to have happened before Gebru sent the email that Dean characterized as an ultimatum.
  - defen 6y ago
    Really makes one wonder if this document is one that Google does not want to come out in discovery, ever, and that it's in some system with a relatively short TTL before it gets deleted, because policy.
mtgx 6y ago
I remember when Google made a whole big deal about their "AI Ethics Board" (or something along those lines), and then not even a year later they reshuffled it because those people were too critical of the company's practices. And then when there was backlash they "promised to do better" and Sundar Pichai came out with some "principles" that the company would follow for AI. Another 1-2 years later and here we are again - this just proves that whatever "AI Ethics Board" they might set-up, it will end up being a sham, because they'd never allow that board to stop them from using AI however they like if it's in the interest of the company's profit growth. If we want real AI oversight we need to demand it from outside nonprofits or even government agencies (why not both?!) - and there should be zero affiliation between the company being monitored and those organizations/agencies.
- tempest_ 6y ago
  At then end of the day the incentives for large companies are always monetary. It might be that they follow ethics because the appearance to do so has a monetary public relations value. It always comes down to that, and for publicly traded companies that set up things like an "AI Ethics Board" it is always for show since the incentives don't allow for anything else. At the end of the day someones compensation depends on these things and you can't be hurting the bottom line.
  - sitkack 6y ago
    This has nothing to do with being a publicly traded company. The founders have a controlling stake.
oh_sigh 6y ago
But Google has published a number of papers that have critiques of the ethics of AI...why would this paper be different?
roenxi 6y ago
Big oil companies don't pump out damning studies on oil use, big tech companies won't pump out research damning the use of their tech. If people have an expectation of Google to turn out academically pure research then I certainly respect the position and encourage it in reality. But thinking like that means life is going to contain a bunch of surprises that really shouldn't be surprising. Google is simply not going to employ people who they recognise as undermining the success of Google. It is not feasible to run a company that way; roughly speaking companies can choose between ruthlessness and bankruptcy. If you expect tolerance of radicals and debate, look to the universities.
- omgwtfbyobbq 6y ago
  I think the difference here is that the research may have shown that Google was unintentionally breaking the law, and after Google realized this, they used an existing review process in a way they don't normally use it to block publication. The possibly shady part is that they could be suppressing evidence that they broke the law, but, like your said, they can decide how to run their own business. I'm not even sure if the researcher would be a whistleblower if they didn't intend to report something illegal. To makes matters worse, in this case at least, the law or laws they may be breaking were established to protect a class of people the researcher is a member of.
- justinclift 6y ago
  > If you expect tolerance of radicals and debate, look to the universities. Universities used to tolerate radicals and debate. But going by the copious media reports of the last few years, that doesn't seem to be how they operate any more.
- Ar-Curunir 6y ago
  Yeah, but then they should get called out for wanting the good PR of having an AI ethics team, but without the headaches of having their poor ethical standards exposed. You can't have it both ways.
- mempko 6y ago
  Companies are given a life under the premise they provide a social good. They get a charter from the government. They are what's called a legal fiction. Government should demand AI ethics instead of letting the companies self regulate. (after all, most AI is developed with gov funds)
sieabahlpark 6y ago
> Ultimately it makes the whole Ethical AI department look more like a rubber stamp for Google. It didn't concern you before?

by acoard 6y ago

orsenthil 6y ago

> Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. I didn't read that. I read the person _demanded_ who said a particular critical feedback, or questioned the approaches instead of addressing them. The person gave the ultimatum to resign if details were not shared.

sudosysgen 6y ago
Unless she lied in her first e-mail, which it doesn't seem like she did, the reason she made those demands is because they asked for a retraction of the paper without indicating why the paper should be retracted. Asking for the identity of people that have the authority to ask for a withdrawal of your research without stating their issues with it seems understandable, if excessive. But maybe I misunderstood something.
- lokar 6y ago
  AIUI, they asked her to retract it because she submitted it before getting final approval, and then they in fact decided not to approve it.
  - Footkerchief 6y ago
    Dean's statement is clear that it was approved before being submitted: > Unfortunately, this particular paper was only shared with a day’s notice before its deadline — we require two weeks for this sort of review — and then instead of awaiting reviewer feedback, it was approved for submission and submitted. A cross functional team then reviewed the paper as part of our regular process and the authors were informed that it didn’t meet our bar for publication and were given feedback about why. [...] We acknowledge that the authors were extremely disappointed with the decision that Megan and I ultimately made, especially as they’d already submitted the paper. There is no statement at all of how to reconcile "approved for submission" with "didn’t meet our bar for publication", which probably means that there is no reconciliation, and the cancellation was done outside normal process.
    - lokar 6y ago
      I see what you mean. I wonder if he is trying to say that there was a process error, it was approved without review (in error), she sent it out, and then they came back to her and said "wait, no, you can't publish that after all"
    - dash2 6y ago
      Sounds like the “editor” or analogous person didn’t wait to hear back from the “referees”. Whose fault this is is not made clear.
UncleMeat 6y ago
If your work is suddenly and unexpected roadblocked at the last second by internal review, the only way to make changes that prevent that from happening in the future is to clearly understand the situations and criticisms that led to the roadblock. This is why understanding who raised these concerns is important. Anonymous feedback blowing up a project at the final moments is sure to frustrate anybody. If what she has said is true then it was also very difficult for her to even have access to the substance of the critique in the first place, with the initial story from her management being that she would not be able to see the documents explaining why the paper was to be retracted. The critique here appears to have been fairly minor, too. Failing to cite some recent research is rarely grounds for rejection.
- siliconmountain 6y ago
  Isn't the academic journal peer review process generally an anonymous feedback mechanism? Why does this need to be different?
  - DonaldPShimoda 6y ago
    Because this isn't peer review — or at least, it's not meant to be (per the top-level comment). That's the whole issue, really: there already exists a peer review process to ensure the paper's academic rigor, so why is Google hiding behind a claim of the necessity of anonymity for a corporate (not academic) process?
  - ethbr0 6y ago
    Because typically your academic journal peers don't work for the same bosses you do.
  - whimsicalism 6y ago
    > academic journal peer review process Fro my understanding, this paper had already passed peer review and been accepted. Google management then decided to block the publication using the IP review process.
    - 6y ago
jonathankoren 6y ago
Given that internal prepublication review at every company I've ever been with is merely there to avoid IP leakage, I find it very hard to believe that the feedback is is given in good faith. It's like the oil industry claiming that a climate change paper isn't talking enough about the economic benefits of growing citrus in Alaska. Quite frankly, there's simply no reason to address them, because the problems with BERT, exist with every LLM. Google stepped in and changed the procedure for this paper, because they wanted to spike it because they were embarrassed by it.

btown 6y ago

Let's be even more clear - they pushed out someone in their Ethical AI department because she wanted to have human conversations to determine the basis for being asked to soften critiques. It's one thing for reviewers, even anonymous reviewers, to reject a paper on its merits; it's another, in Timnit's own words [0], to be told "'it has been decided'" through "a privileged and confidential document to HR" despite clearing the subject matter beforehand. In light of a more general frustration, it's very reasonable for Timnit to escalate the situation by putting her own career on the table, simply to request that people engage with the paper rather than flat-out rejecting it. And if Jeff wants to respond by immediately cutting ties, and by putting out a document that doesn't even address the situation at hand (edit: much less the underlying issues of unequal treatment for women that Timnit describes)... that's a reflection of his ethics and the ethics of the company that stands behind him. [0] For those who haven't read Timnit's memo that Jeff references in the OP, it's worth reading: https://www.platformer.news/p/the-withering-email-that-got-an-ethical https://www.platformer.news/p/the-withering-email-that-got-a... EDIT 2: follow https://twitter.com/timnitGebru https://twitter.com/timnitGebru to see more of her side of the story. She retweeted https://www.wired.com/story/prominent-ai-ethics-researcher-says-google-fired-her/ https://www.wired.com/story/prominent-ai-ethics-researcher-s... as a good explanation of the situation for laypeople.

tfehring 6y ago
Also from Gebru's memo ([0] in the parent comment): >And you are told after a while, that your manager can read you a privileged and confidential document Emphasis mine. Showing your employee that you don't even trust her with a written copy of the rejection of her paper is not a great way to engender a good working relationship. Note that this pretty clearly seems to have happened before Gebru sent the email that Dean characterized as an ultimatum.
- defen 6y ago
  Really makes one wonder if this document is one that Google does not want to come out in discovery, ever, and that it's in some system with a relatively short TTL before it gets deleted, because policy.

mtgx 6y ago

I remember when Google made a whole big deal about their "AI Ethics Board" (or something along those lines), and then not even a year later they reshuffled it because those people were too critical of the company's practices. And then when there was backlash they "promised to do better" and Sundar Pichai came out with some "principles" that the company would follow for AI. Another 1-2 years later and here we are again - this just proves that whatever "AI Ethics Board" they might set-up, it will end up being a sham, because they'd never allow that board to stop them from using AI however they like if it's in the interest of the company's profit growth. If we want real AI oversight we need to demand it from outside nonprofits or even government agencies (why not both?!) - and there should be zero affiliation between the company being monitored and those organizations/agencies.

tempest_ 6y ago
At then end of the day the incentives for large companies are always monetary. It might be that they follow ethics because the appearance to do so has a monetary public relations value. It always comes down to that, and for publicly traded companies that set up things like an "AI Ethics Board" it is always for show since the incentives don't allow for anything else. At the end of the day someones compensation depends on these things and you can't be hurting the bottom line.
- sitkack 6y ago
  This has nothing to do with being a publicly traded company. The founders have a controlling stake.

oh_sigh 6y ago

But Google has published a number of papers that have critiques of the ethics of AI...why would this paper be different?

roenxi 6y ago

Big oil companies don't pump out damning studies on oil use, big tech companies won't pump out research damning the use of their tech. If people have an expectation of Google to turn out academically pure research then I certainly respect the position and encourage it in reality. But thinking like that means life is going to contain a bunch of surprises that really shouldn't be surprising. Google is simply not going to employ people who they recognise as undermining the success of Google. It is not feasible to run a company that way; roughly speaking companies can choose between ruthlessness and bankruptcy. If you expect tolerance of radicals and debate, look to the universities.

omgwtfbyobbq 6y ago
I think the difference here is that the research may have shown that Google was unintentionally breaking the law, and after Google realized this, they used an existing review process in a way they don't normally use it to block publication. The possibly shady part is that they could be suppressing evidence that they broke the law, but, like your said, they can decide how to run their own business. I'm not even sure if the researcher would be a whistleblower if they didn't intend to report something illegal. To makes matters worse, in this case at least, the law or laws they may be breaking were established to protect a class of people the researcher is a member of.
justinclift 6y ago
> If you expect tolerance of radicals and debate, look to the universities. Universities used to tolerate radicals and debate. But going by the copious media reports of the last few years, that doesn't seem to be how they operate any more.
Ar-Curunir 6y ago
Yeah, but then they should get called out for wanting the good PR of having an AI ethics team, but without the headaches of having their poor ethical standards exposed. You can't have it both ways.
mempko 6y ago
Companies are given a life under the premise they provide a social good. They get a charter from the government. They are what's called a legal fiction. Government should demand AI ethics instead of letting the companies self regulate. (after all, most AI is developed with gov funds)

sieabahlpark 6y ago

> Ultimately it makes the whole Ethical AI department look more like a rubber stamp for Google. It didn't concern you before?