6 ms·

It looks like the paper was critical of large language models: https://www.fastcompany.com/90582375/timnit-gebru-google-firing https://www.fastcompany.com/90582

It looks like the paper was critical of large language models: https://www.fastcompany.com/90582375/timnit-gebru-google-firing https://www.fastcompany.com/90582375/timnit-gebru-google-fir... The original article has been updated with Jeff Dean's response, while a member of Gebru's team (who still works at Google Brain) states that Dean's email is misleading and inaccurate: https://twitter.com/alexhanna/status/1334579764573691904 https://twitter.com/alexhanna/status/1334579764573691904 As I understand it: The paper had already been submitted to a conference, and the "cross functional team" Dean speaks of basically reviewed it like peer reviewers and said you have to retract it. It sounds like Gebru and her co-authors were not given an opportunity to respond to the criticisms to stave off retraction. If you've experienced peer review, you know that such reviews are by no means a definitive analysis of a paper, and one usually gets the opportunity to respond. Here, it sounds like anonymous internal peer review was used as a hammer to retract it. Dean's descriptions of the criticisms sounds rather pedestrian to me, as a researcher, and I'm sure Gebru and her co-authors would push back on these characterizations. Honestly, it sounds like the paper was attacking some sacred cows internally. And Gebru's work probably threatened other researchers on other teams. Not unlike traditional peer review mind you, but at least one can usually post in on arXiv without worrying about this.

iandanforth 6y ago
This. Failure to cite relevant research is a reason to reject a paper from a conference but that's not the job of an internal review. Research independence is fundamental and why professors are given tenure. Sure companies are companies and don't play by those rules, but you can't claim to be making a fair academic criticism if you're not adhering to academic norms. If the paper was bad it should have been rejected by the conference to which it was submitted. If the paper was critical of things Google does and it didn't like that, that's pure suppression. And, as you say, academic norms provide a period of rebuttal, it's unclear how much was allowed here. Secondly there is value of having anonymous review in an academic setting, however it sounds like they tried to enforce that anonymous review for the company, which again doesn't serve the same purpose. People should be told exactly why decisions are being made so they can correct their mistakes in the future. If you aren't willing to invite someone into a decision making process, when a large portion of their research is in biases in institutional decision making, then you have got to know you're in for a bad time.
- Laakeri 6y ago
  To me, the Jeff Dean's email sounds like this paper failed to cite some research and as a result made Google look less good than Google actually is. This sounds like somewhat minor academic mistake, but at the same time a major PR mistake.
  - ska 6y ago
    That sounds like something easily rectified, so why push for retracting the submission?
    - fantod 6y ago
      I think the logic is that the paper would need to be retracted, rectified, and then resubmitted. Since the submission deadline was already passed, resubmission would be impossible.
      - ska 6y ago
        Submissions deadline is typically for evaluation, not camera ready. It's not unusual for textual changes to make their way in after that point, either in response to reviewer or to improve the text in a way that doesn't fundamentally change the results.
- titanomachy 6y ago
  I'm not very familiar with the norms of corporate research. Is it normal for corporations to give their researchers tenure-like freedom to publish anything they want? I assume that top researchers (who have their pick of institutions) will choose organizations where they can be relatively free. But what do you mean by "fair academic criticism"? If a company publishes only papers that clearly advance its interests, but the research is sound and independently reproducible, does that bias really lessen the perceived value and quality of the research?
  - ska 6y ago
    At least in theory, something like: Fair academic criticism: We shouldn't publish this paper because it has flaws X,Y,Z. (usually followed by an opportunity for the researchers to address X,Y,Z). Unfair and/or non-academic criticisms: We shouldn't publish this paper because: I don't like the author/authors institution. I don't like this approach, approach X is better. I don't like the conclusions. The last is potentially relevant here. And yes, clearly the quality and value of research is reduced by filtering based on results you want to promote. Which well may be in a companies best interest, but there are sometimes difficult lines to draw - when exactly do you move from "promoting the companies interests" to "propaganda intended to mislead the public"?
lawnchair_larry 6y ago
This Alex Hanna person accusing Dean is certainly not to be taken seriously. I’ll go with Dean over someone with a twitter feed that looks like that. Managing these people must be a nightmare.
- tasuki 6y ago
  What is wrong with Alex Hanna's Twitter feed? To me, it looks like Alex Hanna is very woke, and would probably hold Timnit Gebru's back no matter what - but that's just my guess, and even if so: that doesn't necessarily imply they're wrong. I guess managing people is a nightmare, so managing these people is just a subset of that?
  - njanirudh 6y ago
    When they are overly woke they make accusations without thinking
  - apta 6y ago
    > What is wrong with Alex Hanna's Twitter feed? > is very woke You answered your own question.
- apta 6y ago
  Exactly my thoughts. We as a society went from having dress codes in companies to hiring people that look like freaks.
jeffbee 6y ago
Quite a contrast between Dean's message and Gebru's. I know which one I'd want leading my team.
- chrononaut 6y ago
  I wouldn't necessarily put them on equal footing. One likely had the editorial review of many eyes, while the other was a more personal, empathetic message.
- screye 6y ago
  Jeff Dean is treated like God in these circles alongside the likes of Carmack. Despite that, I'd say he is appropriately rated. There is not a single software team on planet earth that would not want Jeff as their technical leader. It is like comparing a high school athlete to Usain Bolt.
YeGoblynQueenne 6y ago
What I don't understand is why nobody discusses this from the point of view of academic freedom and research integrity. My boss says "retract your paper" and I retract my paper? In what dystopian sci-fi novel? That's just mad.
- visarga 6y ago
  > In what dystopian sci-fi novel? That's just mad. In this world, where there are bills to pay and you can't disadvantage yourself if you want to be ok. Having bills to pay is, I agree, just mad.
- pinewurst 6y ago
  Because this isn't academia, despite the trappings. If I'm working for Reynolds Tobacco Research and wish to publish a paper on tobacco causation of cancer, they're doing to stop my publication under their auspices.
- dekhn 6y ago
  I've published academic research while working at Google. If I got an emergency request to retract a published paper (note that I would have already followed all the necessary steps for publication approval internally), I'd retract it immediately, if there was a legitimate reason (you can see on retractionwatch the sorts of things that lead to retractions). If there wasn't, I'd ask "can we pause the retraction and figure out a better solution based on data", and if the answer was "no", I'd retract it immediately, but follow up with a vigorous protest. Industrial researchers and non-PIs (IE, research analyst working in a PI's lab) don't have the same freedom and integrity rights as a PI. To me this seems like a reasonable (and probably the only reasonable) policy for industry. I don't really think it's dystopian- if a paper I wrote reached the point of needing a retraction it means some serious messup occurred or there's a major PR fiasco. Note that in this case I am not sure if there was a retraction, or an desubmission- the paper hadn't been published yet so it's not really a candidate for retraction. The term for this is 'withdraw submission'
- PeterisP 6y ago
  There's no academic freedom expected - that's private R&D work, the research and the paper is done on behalf of the company and wholly owned by it, not the employees who did it, and so your boss naturally has full control on when, how, where and if that research should be published. It's just as if you'd want to open source some code that you wrote for your job, and suddenly the boss says "hey, don't do that" - there are many reasons why they might want to not publish it, and it's not your code to publish as you want.
anonvenger 6y ago
From Jeff Dean's email: > Timnit co-authored a paper with four fellow Googlers as well as some external collaborators that needed to go through our review process (as is the case with all externally submitted papers). We’ve approved dozens of papers that Timnit and/or the other Googlers have authored and then published, but as you know, papers often require changes during the internal review process (or are even deemed unsuitable for submission). Unfortunately, this particular paper was only shared with a day’s notice before its deadline — we require two weeks for this sort of review — and then instead of awaiting reviewer feedback, it was approved for submission and submitted. > A cross functional team then reviewed the paper as part of our regular process and the authors were informed that it didn’t meet our bar for publication and were given feedback about why. It ignored too much relevant research — for example, it talked about the environmental impact of large models, but disregarded subsequent research showing much greater efficiencies. Similarly, it raised concerns about bias in language models, but didn’t take into account recent research to mitigate these issues. We acknowledge that the authors were extremely disappointed with the decision that Megan and I ultimately made, especially as they’d already submitted the paper. > Timnit responded with an email requiring that a number of conditions be met in order for her to continue working at Google, including revealing the identities of every person who Megan and I had spoken to and consulted as part of the review of the paper and the exact feedback. Timnit wrote that if we didn’t meet these demands, she would leave Google and work on an end date. We accept and respect her decision to resign from Google.
- ZephyrBlu 6y ago
  That seems pretty damning to me. Especially considering Timnit is not sharing what her demands were. Google went through their standard process, and the paper was rejected. If they have actually approved dozens of Timnit's papers before, then the likelihood that this rejection is due to discrimination seems low.
  - deleted 6y ago
    [deleted]
- tanilama 6y ago
  > revealing the identities of every person who Megan and I had spoken to Big red flag. Why are the identities of reviewers important here? Did she plan to take those reviewers to the court of public opinions for a trial/exposure? A lesson is delivered in time IMO. Those entitled people need to be called out.
apta 6y ago
> while a member of Gebru's team I have a hard time taking someone who looks like that seriously. Says a lot about her and her team.
somepersoninny 6y ago
Her exit from the company was not due to her research. It was for her behavior and her attitude. The paper was just the catalyst that kicked everything else off. Why did they not fight to keep her, if she was so singularly brilliant? IMO, it's because she wasn't an employee they wanted to keep around. People don't get fired for asking for opportunities to respond to criticisms. My guess is there were previous incidences regarding her behavior and, in this situation, she threatened to quit and they accepted.

iandanforth 6y ago

This. Failure to cite relevant research is a reason to reject a paper from a conference but that's not the job of an internal review. Research independence is fundamental and why professors are given tenure. Sure companies are companies and don't play by those rules, but you can't claim to be making a fair academic criticism if you're not adhering to academic norms. If the paper was bad it should have been rejected by the conference to which it was submitted. If the paper was critical of things Google does and it didn't like that, that's pure suppression. And, as you say, academic norms provide a period of rebuttal, it's unclear how much was allowed here. Secondly there is value of having anonymous review in an academic setting, however it sounds like they tried to enforce that anonymous review for the company, which again doesn't serve the same purpose. People should be told exactly why decisions are being made so they can correct their mistakes in the future. If you aren't willing to invite someone into a decision making process, when a large portion of their research is in biases in institutional decision making, then you have got to know you're in for a bad time.

Laakeri 6y ago
To me, the Jeff Dean's email sounds like this paper failed to cite some research and as a result made Google look less good than Google actually is. This sounds like somewhat minor academic mistake, but at the same time a major PR mistake.
- ska 6y ago
  That sounds like something easily rectified, so why push for retracting the submission?
  - fantod 6y ago
    I think the logic is that the paper would need to be retracted, rectified, and then resubmitted. Since the submission deadline was already passed, resubmission would be impossible.
    - ska 6y ago
      Submissions deadline is typically for evaluation, not camera ready. It's not unusual for textual changes to make their way in after that point, either in response to reviewer or to improve the text in a way that doesn't fundamentally change the results.
titanomachy 6y ago
I'm not very familiar with the norms of corporate research. Is it normal for corporations to give their researchers tenure-like freedom to publish anything they want? I assume that top researchers (who have their pick of institutions) will choose organizations where they can be relatively free. But what do you mean by "fair academic criticism"? If a company publishes only papers that clearly advance its interests, but the research is sound and independently reproducible, does that bias really lessen the perceived value and quality of the research?
- ska 6y ago
  At least in theory, something like: Fair academic criticism: We shouldn't publish this paper because it has flaws X,Y,Z. (usually followed by an opportunity for the researchers to address X,Y,Z). Unfair and/or non-academic criticisms: We shouldn't publish this paper because: I don't like the author/authors institution. I don't like this approach, approach X is better. I don't like the conclusions. The last is potentially relevant here. And yes, clearly the quality and value of research is reduced by filtering based on results you want to promote. Which well may be in a companies best interest, but there are sometimes difficult lines to draw - when exactly do you move from "promoting the companies interests" to "propaganda intended to mislead the public"?

lawnchair_larry 6y ago

This Alex Hanna person accusing Dean is certainly not to be taken seriously. I’ll go with Dean over someone with a twitter feed that looks like that. Managing these people must be a nightmare.

tasuki 6y ago
What is wrong with Alex Hanna's Twitter feed? To me, it looks like Alex Hanna is very woke, and would probably hold Timnit Gebru's back no matter what - but that's just my guess, and even if so: that doesn't necessarily imply they're wrong. I guess managing people is a nightmare, so managing these people is just a subset of that?
- njanirudh 6y ago
  When they are overly woke they make accusations without thinking
- apta 6y ago
  > What is wrong with Alex Hanna's Twitter feed? > is very woke You answered your own question.
apta 6y ago
Exactly my thoughts. We as a society went from having dress codes in companies to hiring people that look like freaks.

jeffbee 6y ago

Quite a contrast between Dean's message and Gebru's. I know which one I'd want leading my team.

chrononaut 6y ago
I wouldn't necessarily put them on equal footing. One likely had the editorial review of many eyes, while the other was a more personal, empathetic message.
screye 6y ago
Jeff Dean is treated like God in these circles alongside the likes of Carmack. Despite that, I'd say he is appropriately rated. There is not a single software team on planet earth that would not want Jeff as their technical leader. It is like comparing a high school athlete to Usain Bolt.

YeGoblynQueenne 6y ago

What I don't understand is why nobody discusses this from the point of view of academic freedom and research integrity. My boss says "retract your paper" and I retract my paper? In what dystopian sci-fi novel? That's just mad.

visarga 6y ago
> In what dystopian sci-fi novel? That's just mad. In this world, where there are bills to pay and you can't disadvantage yourself if you want to be ok. Having bills to pay is, I agree, just mad.
pinewurst 6y ago
Because this isn't academia, despite the trappings. If I'm working for Reynolds Tobacco Research and wish to publish a paper on tobacco causation of cancer, they're doing to stop my publication under their auspices.
dekhn 6y ago
I've published academic research while working at Google. If I got an emergency request to retract a published paper (note that I would have already followed all the necessary steps for publication approval internally), I'd retract it immediately, if there was a legitimate reason (you can see on retractionwatch the sorts of things that lead to retractions). If there wasn't, I'd ask "can we pause the retraction and figure out a better solution based on data", and if the answer was "no", I'd retract it immediately, but follow up with a vigorous protest. Industrial researchers and non-PIs (IE, research analyst working in a PI's lab) don't have the same freedom and integrity rights as a PI. To me this seems like a reasonable (and probably the only reasonable) policy for industry. I don't really think it's dystopian- if a paper I wrote reached the point of needing a retraction it means some serious messup occurred or there's a major PR fiasco. Note that in this case I am not sure if there was a retraction, or an desubmission- the paper hadn't been published yet so it's not really a candidate for retraction. The term for this is 'withdraw submission'
PeterisP 6y ago
There's no academic freedom expected - that's private R&D work, the research and the paper is done on behalf of the company and wholly owned by it, not the employees who did it, and so your boss naturally has full control on when, how, where and if that research should be published. It's just as if you'd want to open source some code that you wrote for your job, and suddenly the boss says "hey, don't do that" - there are many reasons why they might want to not publish it, and it's not your code to publish as you want.

anonvenger 6y ago

From Jeff Dean's email: > Timnit co-authored a paper with four fellow Googlers as well as some external collaborators that needed to go through our review process (as is the case with all externally submitted papers). We’ve approved dozens of papers that Timnit and/or the other Googlers have authored and then published, but as you know, papers often require changes during the internal review process (or are even deemed unsuitable for submission). Unfortunately, this particular paper was only shared with a day’s notice before its deadline — we require two weeks for this sort of review — and then instead of awaiting reviewer feedback, it was approved for submission and submitted. > A cross functional team then reviewed the paper as part of our regular process and the authors were informed that it didn’t meet our bar for publication and were given feedback about why. It ignored too much relevant research — for example, it talked about the environmental impact of large models, but disregarded subsequent research showing much greater efficiencies. Similarly, it raised concerns about bias in language models, but didn’t take into account recent research to mitigate these issues. We acknowledge that the authors were extremely disappointed with the decision that Megan and I ultimately made, especially as they’d already submitted the paper. > Timnit responded with an email requiring that a number of conditions be met in order for her to continue working at Google, including revealing the identities of every person who Megan and I had spoken to and consulted as part of the review of the paper and the exact feedback. Timnit wrote that if we didn’t meet these demands, she would leave Google and work on an end date. We accept and respect her decision to resign from Google.

ZephyrBlu 6y ago
That seems pretty damning to me. Especially considering Timnit is not sharing what her demands were. Google went through their standard process, and the paper was rejected. If they have actually approved dozens of Timnit's papers before, then the likelihood that this rejection is due to discrimination seems low.
- deleted 6y ago
  [deleted]
tanilama 6y ago
> revealing the identities of every person who Megan and I had spoken to Big red flag. Why are the identities of reviewers important here? Did she plan to take those reviewers to the court of public opinions for a trial/exposure? A lesson is delivered in time IMO. Those entitled people need to be called out.

apta 6y ago

> while a member of Gebru's team I have a hard time taking someone who looks like that seriously. Says a lot about her and her team.

somepersoninny 6y ago

Her exit from the company was not due to her research. It was for her behavior and her attitude. The paper was just the catalyst that kicked everything else off. Why did they not fight to keep her, if she was so singularly brilliant? IMO, it's because she wasn't an employee they wanted to keep around. People don't get fired for asking for opportunities to respond to criticisms. My guess is there were previous incidences regarding her behavior and, in this situation, she threatened to quit and they accepted.