6 ms·

I don't know Jeff Dean. I have read some of his work, watched some of his presentations. He seems a credible bloke. This, though, looks and feels like thinly

I don't know Jeff Dean. I have read some of his work, watched some of his presentations. He seems a credible bloke. This, though, looks and feels like thinly-veiled retroactive and pretty unconvincing PR. It's short on detail and appears somewhat at odds with several points from Timnit Gebru's resignation note [0]: - Dean says the paper was reviewed by a "cross-functional team". Gebru says she received the feedback through a "privileged and confidential document to HR" - Dean says the paper was submitted for review on the day it was due to be published; Gebru says they had notified "PR & Policy 2 months before". - Dean suggests the feedback was due to the paper not highlighting mitigating work for some of the limitations the paper was exposing. That seems like a very normal part of the research process. Why, then, does Gebru claim that she was told that a "manager can read you a privileged and confidential document" and that no other recourse or exploration of the feedback was permissible? The only thing we know from the outside is that reality will be far more nuanced and complicated than the tidbits that leak out. Even allowing for that though - and reading some of the related comments here - Google isn't coming out of this well at the moment. [0]: https://www.platformer.news/p/the-withering-email-that-got-an-ethical https://www.platformer.news/p/the-withering-email-that-got-a... EDIT: Fixed spelling of Timnit Gebru's name.

sillysaurusx 6y ago
Dean says the paper was submitted for review on the day it was due to be published; Gebru says they had notified "PR & Policy 2 months before". Wait, really? That’s an important detail if true. The one-day timeline was a central part of the narrative surrounding this story. Notifying them two months ahead of time makes this a completely different situation. I’m a bit skeptical. Could this claim be verified somehow? Since it’s very public news at this point, we may as well try to be rigorous.
- usmannk 6y ago
  From what I read it sounds like Timnit cleared the general idea of the paper 2 months before. The paper itself seems to have been submitted for approval 1 day prior to the deadline. Another HN commenter says that submitting papers for approval "hours" before the deadline is common at Google.
  - sillysaurusx 6y ago
    Thank you. I’m not sure what to think. It seemed absurd to submit a paper for internal academic review one day prior to a major deadline. Yet today we’re hearing that it’s common (I can believe that; standard big company stuff) and that Google hasn’t been doing academic reviews at all until very recently / possibly this incident. So it feels like, suddenly, the cornerstones of the arguments against her are vaporizing before our eyes. This could go badly for El Goog unless they stop making official statements on the matter. Saying nothing would have been better than giving a convenient post for all the former Google employees to come out of the woodwork and say “That’s not true! Google never did academic reviews; they solely checked whether business IP was being exposed.” It’s ironic that people are painting her as unprofessional in that context; I’d be frustrated too, if that’s really the situation.
    - whimsicalism 6y ago
      > It seemed absurd to submit a paper for internal academic review one day prior to a major deadline I think that there isn't internal academic review of the sort implied by Jeff at Google. Timnit seems pretty clearly in the right here. As an AI researcher at a competitor, this impacts my desire to join Google in the future. I imagine these sort of PR disasters hurt their standing in the academic labor market.
      - lokar 6y ago
        > I think that there isn't internal academic review of the sort implied by Jeff at Google. Why do you think that?
        sillysaurusx 6y ago
        I'm interested to know too. Because although I treat my ML work professionally, I haven't worked at a large AI lab. So I had no idea whether academic review was common inside DeepMind/OpenAI/Facebook AI, or whether researchers generally did their own thing -- i.e. that the researcher and their co-authors are the main judges of their own academic integrity. And in hindsight, it seems dumb to think it'd be any other way. Of course the researchers are their own judge. That's what the reviewers are for! You submit your paper for academic review at a journal, and the reviewers are in charge of reviewing. Would you really want to mess with that dynamic if you're a big company? It's been a tried-and-true way to do science for more than a century. It's also a recipe for failing at science, as many will attest. But being allowed to fail at science is a key aspect of science. It would be terrible if we only published papers that were completely correct in every detail, because it means everyone is playing it safe rather than pushing the boundaries. The most interesting work is usually on the frontier of some new idea. When the news broke, I didn't give it a second thought. "Oh, Jeff is saying that there's an academic review process. Yeah, obviously DeepMind would have something like that. And what's this -- she sent the paper one day before the journal deadline? That's almost giving them the middle finger. Yeah, pretty clear-cut firing." ... But when you think back on it, none of that adds up. Researchers are paid to do research. Being hamstrung by some manager insisting that you namedrop every relevant paper from the last decade would certainly be rigorous, but not necessarily productive. Sure, you can argue that maybe she should have talked about X or Y. But you could also write your own paper. I'll admit, I didn't think highly of her. All I knew was that she liked to stir up drama. Why won't she just keep quiet and do her job like everyone else? Yet now it seems like she was doing her job. And if I ask myself how I'd react in that situation -- some middle manager is forcing a bogus new "review" process, and now they're demanding us to retract a paper that we put several months of work into, for reasons other than "You're revealing Google's IP," then my thoughts would be: (a) where were you during the two months I've been writing this and asking for feedback? (b) what are you trying to accomplish here, and is this really how a world-class institution treats the process? Every company is different. And at Google scale, different teams are different. But now it's looking pretty bad. They certainly had grounds to fire her, and for many folks perhaps that's enough, but as a researcher I'm thinking "Why did Google try so hard to retract her paper anyway?" They keep dancing around that. And the article certainly doesn't address it: Our aim is to rival peer-reviewed journals in terms of the rigor and thoughtfulness in how we review research before publication. Why? That's the point of publishing! Do reviewers just say "Oh this is from Google" and click the "approve" button? Maybe, but the whole point is for people to read a paper and decide for themselves whether it's mistaken. This whole "keep it under the rug until it's polished perfectly for six months for no reason other than prestige" is... well, rather a grim-sounding idea. Outsiders can't know what insiders know. But we can picture various things based on the information we're getting. And this reads like some manager tried to double down, and she called him on it each time. After four or five doublings, now it's headline news and Google is looking like they went nuclear without some very solid reasons.
mdanger007 6y ago
Your honor, I didn't beat my wife. But if I did, she deserved it.
tfehring 6y ago
>Dean says the paper was reviewed by a "cross-functional team". Gebru says she received the feedback through a "privileged and confidential document to HR" I don't think these two takes are at odds with each other at all. It sounds like a cross-functional team reviewed the paper and produced that "privileged and confidential document" but Gebru didn't find that document sufficiently detailed.
kevinskii 6y ago
> Gebru says she received the feedback through a "privileged and confidential document to HR" I agree that Google isn't looking good at the moment. But if I had a colleague who seemed intent on finding ways to place the company in legal jeopardy, then I too would avoid direct communication whenever possible.

sillysaurusx 6y ago

Dean says the paper was submitted for review on the day it was due to be published; Gebru says they had notified "PR & Policy 2 months before". Wait, really? That’s an important detail if true. The one-day timeline was a central part of the narrative surrounding this story. Notifying them two months ahead of time makes this a completely different situation. I’m a bit skeptical. Could this claim be verified somehow? Since it’s very public news at this point, we may as well try to be rigorous.

usmannk 6y ago
From what I read it sounds like Timnit cleared the general idea of the paper 2 months before. The paper itself seems to have been submitted for approval 1 day prior to the deadline. Another HN commenter says that submitting papers for approval "hours" before the deadline is common at Google.
- sillysaurusx 6y ago
  Thank you. I’m not sure what to think. It seemed absurd to submit a paper for internal academic review one day prior to a major deadline. Yet today we’re hearing that it’s common (I can believe that; standard big company stuff) and that Google hasn’t been doing academic reviews at all until very recently / possibly this incident. So it feels like, suddenly, the cornerstones of the arguments against her are vaporizing before our eyes. This could go badly for El Goog unless they stop making official statements on the matter. Saying nothing would have been better than giving a convenient post for all the former Google employees to come out of the woodwork and say “That’s not true! Google never did academic reviews; they solely checked whether business IP was being exposed.” It’s ironic that people are painting her as unprofessional in that context; I’d be frustrated too, if that’s really the situation.
  - whimsicalism 6y ago
    > It seemed absurd to submit a paper for internal academic review one day prior to a major deadline I think that there isn't internal academic review of the sort implied by Jeff at Google. Timnit seems pretty clearly in the right here. As an AI researcher at a competitor, this impacts my desire to join Google in the future. I imagine these sort of PR disasters hurt their standing in the academic labor market.
    - lokar 6y ago
      > I think that there isn't internal academic review of the sort implied by Jeff at Google. Why do you think that?
      - sillysaurusx 6y ago
        I'm interested to know too. Because although I treat my ML work professionally, I haven't worked at a large AI lab. So I had no idea whether academic review was common inside DeepMind/OpenAI/Facebook AI, or whether researchers generally did their own thing -- i.e. that the researcher and their co-authors are the main judges of their own academic integrity. And in hindsight, it seems dumb to think it'd be any other way. Of course the researchers are their own judge. That's what the reviewers are for! You submit your paper for academic review at a journal, and the reviewers are in charge of reviewing. Would you really want to mess with that dynamic if you're a big company? It's been a tried-and-true way to do science for more than a century. It's also a recipe for failing at science, as many will attest. But being allowed to fail at science is a key aspect of science. It would be terrible if we only published papers that were completely correct in every detail, because it means everyone is playing it safe rather than pushing the boundaries. The most interesting work is usually on the frontier of some new idea. When the news broke, I didn't give it a second thought. "Oh, Jeff is saying that there's an academic review process. Yeah, obviously DeepMind would have something like that. And what's this -- she sent the paper one day before the journal deadline? That's almost giving them the middle finger. Yeah, pretty clear-cut firing." ... But when you think back on it, none of that adds up. Researchers are paid to do research. Being hamstrung by some manager insisting that you namedrop every relevant paper from the last decade would certainly be rigorous, but not necessarily productive. Sure, you can argue that maybe she should have talked about X or Y. But you could also write your own paper. I'll admit, I didn't think highly of her. All I knew was that she liked to stir up drama. Why won't she just keep quiet and do her job like everyone else? Yet now it seems like she was doing her job. And if I ask myself how I'd react in that situation -- some middle manager is forcing a bogus new "review" process, and now they're demanding us to retract a paper that we put several months of work into, for reasons other than "You're revealing Google's IP," then my thoughts would be: (a) where were you during the two months I've been writing this and asking for feedback? (b) what are you trying to accomplish here, and is this really how a world-class institution treats the process? Every company is different. And at Google scale, different teams are different. But now it's looking pretty bad. They certainly had grounds to fire her, and for many folks perhaps that's enough, but as a researcher I'm thinking "Why did Google try so hard to retract her paper anyway?" They keep dancing around that. And the article certainly doesn't address it: Our aim is to rival peer-reviewed journals in terms of the rigor and thoughtfulness in how we review research before publication. Why? That's the point of publishing! Do reviewers just say "Oh this is from Google" and click the "approve" button? Maybe, but the whole point is for people to read a paper and decide for themselves whether it's mistaken. This whole "keep it under the rug until it's polished perfectly for six months for no reason other than prestige" is... well, rather a grim-sounding idea. Outsiders can't know what insiders know. But we can picture various things based on the information we're getting. And this reads like some manager tried to double down, and she called him on it each time. After four or five doublings, now it's headline news and Google is looking like they went nuclear without some very solid reasons.

mdanger007 6y ago

Your honor, I didn't beat my wife. But if I did, she deserved it.

tfehring 6y ago

>Dean says the paper was reviewed by a "cross-functional team". Gebru says she received the feedback through a "privileged and confidential document to HR" I don't think these two takes are at odds with each other at all. It sounds like a cross-functional team reviewed the paper and produced that "privileged and confidential document" but Gebru didn't find that document sufficiently detailed.

kevinskii 6y ago

> Gebru says she received the feedback through a "privileged and confidential document to HR" I agree that Google isn't looking good at the moment. But if I had a colleague who seemed intent on finding ways to place the company in legal jeopardy, then I too would avoid direct communication whenever possible.