5 ms·

After reading Jeff Dean's response, I can only come to the conclusion that Timnit Gebru acted like a primadonna. It is completely normal to have to obtain prepu

After reading Jeff Dean's response, I can only come to the conclusion that Timnit Gebru acted like a primadonna. It is completely normal to have to obtain prepublication signoff on material before it is submitted to a conference (and manager signoff even before abstract submission). Given the breadth of experience at Google, it seems strange not to avail yourself of this. Demanding to know the identity of reviewers is absurd (no journal would tolerate that) and deeply unprofessional. Making it an explicit ultimatum was her decision. Denigrating the entire area of research at Google on a large mailing list is the action of someone who wants to be terminated. People claiming that the deficiencies in the paper are minor and wouldn't be blockers obviously have little experience submitting to academic journals. Other parts of Google doing deeply technical work probably don't have the same level of review as the Ethical AI group -- for obvious reasons. There is usually a long back and forth -- there are even memes about the infuriating comments from "Reviewer 2" [1][2]. Omitting to mention argument-obsoleting developments in the field (from your own lab!) is more than enough to send you back to extensive redrafting. To be clear on terminology -- a retraction is an academic black mark, and occurs to a paper after publication, usually for reasons of research misconduct. This is not an instance of that. [1] https://twitter.com/redpenblackpen/status/1133440569907195904/photo/1 https://twitter.com/redpenblackpen/status/113344056990719590... [2] http://jasonya.com/wp/wp-content/uploads/2016/01/PowerResponsibility.jpg http://jasonya.com/wp/wp-content/uploads/2016/01/PowerRespon...

joshuamorton 6y ago
> It is completely normal to have to obtain prepublication signoff on material before it is submitted to a conference (and manager signoff even before abstract submission) To be clear, both of these had been received for the paper in question.
- angry_octet 6y ago
  That isn't the impression I had from reading the principal's statements, to wit: > Unfortunately, this particular paper was only shared with a day’s notice before its deadline — we require two weeks for this sort of review — and then instead of awaiting reviewer feedback, it was approved for submission and submitted. It seems like someone short-circuited the review process and submitted without review to meet a deadline, with a post-submission review. When this occurred, it was not all green lights. The expectation is that you then pull the submission and, post review, submit elsewhere. After all, conferences are all virtual now, it is not a hardship to submit elsewhere. If you have other information as to the facts or the internal process do tell.
  - Footkerchief 6y ago
    How do you interpret "approved for submission" as "submitted without review"?
    - abeyer 6y ago
      Perhaps by not eliding the first part of "instead of awaiting reviewer feedback, it was approved for submission."
  - joshuamorton 6y ago
    Jeff's statement is a lie. I work at Google, and the publication approval process doesn't normally take to weeks. Any Google employee can verify this, and others have, both in this thread and on twitter. There's some documentation that asks for 2 weeks, but it's not followed, and inconsistent application of policy is absolutely a concern in a case like this. A summary of the events, as best as I can tell, is the normal presubmission review was done. After the paper was approved and submitted, someone, whether this was upper management or some other entity, did some additional review and required that the authors withdraw (?, but retract is the wrong term) the paper. From what I know, both firsthand and from other sources[0], while not spotless, the issues with the paper were mostly nitpicks and fairly straightforward to resolve. That they weren't even provided until some escalation from Dr. Gebru is strange. That even after they were, sort of, provided, she and her team were not given the option to address them in the paper, is extraordinarily strange. [0]: https://www.reddit.com/r/MachineLearning/comments/k69eq0/n_the_abstract_of_the_paper_that_led_to_timnit/ https://www.reddit.com/r/MachineLearning/comments/k69eq0/n_t...
    - angry_octet 6y ago
      But this supposes that there is some deep and nefarious need for Google to kill a paper written by some AI Ethics people. From looking at their recent pubs I can't think of anything remotely significant about any one paper they could write. Hopefully we'll get a full leak at some point.
      - 6y ago
mattdeboard 6y ago
I think, given the top comment on this post, your comment is far too credulous about Google's incentives being aligned with scientific rigor alone.
- angry_octet 6y ago
  To be sure, Google has other objectives. But the people in this area are essentially academics, and I'd expect them to have a similar process given their backgrounds. The papers from Google are routinely of high calibre. The non-research motive is arguably to control the narrative about using ML (I admit I still choke at calling it AI) and big data techniques. I fail to see how this is advanced by having a very public spat. I've no doubt some senior exec decided she had to go, but I don't think it is because of any intersectional reason, or to cover up any particular publication, but that she wasn't seen as an asset to the company. Strategies for negotiating exercise of power are very different at Google to social sciences academia (or twitter, which is engorged with righteousness over this), which she seems not to have grasped. There are few enfants terribles in corporate tech that don't have controlling stock.
  - diffserv 6y ago
    This. And one of the main reasons that the paper from Google are routinely of high caliber is due to the internal review processes and your peers. Publishing a piece of work for the sake of publishing by ignoring the processes that are put in place is outright irresponsible and that's the end of it.
bmitc 6y ago
Gebru's email to team members: https://www.platformer.news/p/the-withering-email-that-got-an-ethical https://www.platformer.news/p/the-withering-email-that-got-a... Tweet explaining how she "resigned": https://twitter.com/timnitGebru/status/1334343577044979712?s=20 https://twitter.com/timnitGebru/status/1334343577044979712?s... An external reviewer of the paper: https://www.reddit.com/r/MachineLearning/comments/k69eq0/n_the_abstract_of_the_paper_that_led_to_timnit/gejt4c0/ https://www.reddit.com/r/MachineLearning/comments/k69eq0/n_t... I think you should be more open about what potentially actually happened because your first sentence seems very accusatory without proper information, the very bias that Gebru speaks of. According to her email that Dean references in his, her "feedback" for the paper was given in the form of a confidential HR document. That seems highly nonstandard for a review process supposedly only concerned with scientific rigor and is not addressed at all by Dean's statement. Further, his statement clearly, likely intentionally, muddies the water about what the actual sequence of events were that led to the paper being created, internal reviewers originally being notified, when the submission happened, who approved it, who submitted it externally, who asked for this to be retracted, and when and in what order all this happened. The fact that his statement, that was apparently meant to clarify these exact proceedings, makes it even more vague about what actually happened seems pretty damning to Dean's case, in my opinion. Both Gebru's and Dean's statements have very little overlap in flavor and in facts, so it's pretty apparent something is going on here that is nontrivial and abnormal.
1234fdsazxcv 6y ago
I would agree with this assessment. It seems people think this is like academia -- it's not. There are no guarantees of tenure at a private company, even one with a research culture. If she wants that, then give up the paycheck and go back to academia. Google doesn't owe her anything. (I bet she'll get a decent settlement.) AI ethics are important and I'm willing to bet that the Google execs want to know about biases and other problems in their products and probably don't care that much what is published. They probably don't want high maintenance researchers that cause them headaches and are obsessed with microaggressions, etc. I expect that this stuff has been a distraction for too long and they are happy to be rid of this and happy for the signal it sends across the company. I have little sympathy or concern for Ivy educated elite in-fighting. There is probably a huge number of qualified people just dying to have the opportunity to work there, have access to those resources and do research.
gowld 6y ago
Whichever side's account you read, that side looks correct. This is not surprising. There's a classic documentary about this phenomenon: https://www.youtube.com/watch?v=9UiF7BsC1Ig https://www.youtube.com/watch?v=9UiF7BsC1Ig
- angry_octet 6y ago
  That's a very snide comment. I've read her tweets and other Brain tweets (eg, from mmitchell_ai, though she deleted them, which seems wise as they were angry). I don't take any of them at their surface as being truthful. I think we're only seeing the 10% of the iceberg. But I don't find Gebru convincing.

joshuamorton 6y ago

> It is completely normal to have to obtain prepublication signoff on material before it is submitted to a conference (and manager signoff even before abstract submission) To be clear, both of these had been received for the paper in question.

angry_octet 6y ago
That isn't the impression I had from reading the principal's statements, to wit: > Unfortunately, this particular paper was only shared with a day’s notice before its deadline — we require two weeks for this sort of review — and then instead of awaiting reviewer feedback, it was approved for submission and submitted. It seems like someone short-circuited the review process and submitted without review to meet a deadline, with a post-submission review. When this occurred, it was not all green lights. The expectation is that you then pull the submission and, post review, submit elsewhere. After all, conferences are all virtual now, it is not a hardship to submit elsewhere. If you have other information as to the facts or the internal process do tell.
- Footkerchief 6y ago
  How do you interpret "approved for submission" as "submitted without review"?
  - abeyer 6y ago
    Perhaps by not eliding the first part of "instead of awaiting reviewer feedback, it was approved for submission."
- joshuamorton 6y ago
  Jeff's statement is a lie. I work at Google, and the publication approval process doesn't normally take to weeks. Any Google employee can verify this, and others have, both in this thread and on twitter. There's some documentation that asks for 2 weeks, but it's not followed, and inconsistent application of policy is absolutely a concern in a case like this. A summary of the events, as best as I can tell, is the normal presubmission review was done. After the paper was approved and submitted, someone, whether this was upper management or some other entity, did some additional review and required that the authors withdraw (?, but retract is the wrong term) the paper. From what I know, both firsthand and from other sources[0], while not spotless, the issues with the paper were mostly nitpicks and fairly straightforward to resolve. That they weren't even provided until some escalation from Dr. Gebru is strange. That even after they were, sort of, provided, she and her team were not given the option to address them in the paper, is extraordinarily strange. [0]: https://www.reddit.com/r/MachineLearning/comments/k69eq0/n_the_abstract_of_the_paper_that_led_to_timnit/ https://www.reddit.com/r/MachineLearning/comments/k69eq0/n_t...
  - angry_octet 6y ago
    But this supposes that there is some deep and nefarious need for Google to kill a paper written by some AI Ethics people. From looking at their recent pubs I can't think of anything remotely significant about any one paper they could write. Hopefully we'll get a full leak at some point.
    - 6y ago

mattdeboard 6y ago

I think, given the top comment on this post, your comment is far too credulous about Google's incentives being aligned with scientific rigor alone.

angry_octet 6y ago
To be sure, Google has other objectives. But the people in this area are essentially academics, and I'd expect them to have a similar process given their backgrounds. The papers from Google are routinely of high calibre. The non-research motive is arguably to control the narrative about using ML (I admit I still choke at calling it AI) and big data techniques. I fail to see how this is advanced by having a very public spat. I've no doubt some senior exec decided she had to go, but I don't think it is because of any intersectional reason, or to cover up any particular publication, but that she wasn't seen as an asset to the company. Strategies for negotiating exercise of power are very different at Google to social sciences academia (or twitter, which is engorged with righteousness over this), which she seems not to have grasped. There are few enfants terribles in corporate tech that don't have controlling stock.
- diffserv 6y ago
  This. And one of the main reasons that the paper from Google are routinely of high caliber is due to the internal review processes and your peers. Publishing a piece of work for the sake of publishing by ignoring the processes that are put in place is outright irresponsible and that's the end of it.

bmitc 6y ago

Gebru's email to team members: https://www.platformer.news/p/the-withering-email-that-got-an-ethical https://www.platformer.news/p/the-withering-email-that-got-a... Tweet explaining how she "resigned": https://twitter.com/timnitGebru/status/1334343577044979712?s=20 https://twitter.com/timnitGebru/status/1334343577044979712?s... An external reviewer of the paper: https://www.reddit.com/r/MachineLearning/comments/k69eq0/n_the_abstract_of_the_paper_that_led_to_timnit/gejt4c0/ https://www.reddit.com/r/MachineLearning/comments/k69eq0/n_t... I think you should be more open about what potentially actually happened because your first sentence seems very accusatory without proper information, the very bias that Gebru speaks of. According to her email that Dean references in his, her "feedback" for the paper was given in the form of a confidential HR document. That seems highly nonstandard for a review process supposedly only concerned with scientific rigor and is not addressed at all by Dean's statement. Further, his statement clearly, likely intentionally, muddies the water about what the actual sequence of events were that led to the paper being created, internal reviewers originally being notified, when the submission happened, who approved it, who submitted it externally, who asked for this to be retracted, and when and in what order all this happened. The fact that his statement, that was apparently meant to clarify these exact proceedings, makes it even more vague about what actually happened seems pretty damning to Dean's case, in my opinion. Both Gebru's and Dean's statements have very little overlap in flavor and in facts, so it's pretty apparent something is going on here that is nontrivial and abnormal.

1234fdsazxcv 6y ago

I would agree with this assessment. It seems people think this is like academia -- it's not. There are no guarantees of tenure at a private company, even one with a research culture. If she wants that, then give up the paycheck and go back to academia. Google doesn't owe her anything. (I bet she'll get a decent settlement.) AI ethics are important and I'm willing to bet that the Google execs want to know about biases and other problems in their products and probably don't care that much what is published. They probably don't want high maintenance researchers that cause them headaches and are obsessed with microaggressions, etc. I expect that this stuff has been a distraction for too long and they are happy to be rid of this and happy for the signal it sends across the company. I have little sympathy or concern for Ivy educated elite in-fighting. There is probably a huge number of qualified people just dying to have the opportunity to work there, have access to those resources and do research.

gowld 6y ago

Whichever side's account you read, that side looks correct. This is not surprising. There's a classic documentary about this phenomenon: https://www.youtube.com/watch?v=9UiF7BsC1Ig https://www.youtube.com/watch?v=9UiF7BsC1Ig

angry_octet 6y ago
That's a very snide comment. I've read her tweets and other Brain tweets (eg, from mmitchell_ai, though she deleted them, which seems wise as they were angry). I don't take any of them at their surface as being truthful. I think we're only seeing the 10% of the iceberg. But I don't find Gebru convincing.