5 ms·

Re the existence of an internal review (tl;dr: it definitely does exist!): A Google employee here who's actually published research there. There definitely IS a

Re the existence of an internal review (tl;dr: it definitely does exist!): A Google employee here who's actually published research there. There definitely IS an internal review and an approval process and every single paper I have published while there had to go through it. Everyone I knew had to submit for an internal review well in advance and I personally couldn't even submit to a conference review once because we only asked for the internal review 3 days in advance, which wasn't enough time. My understanding was that everyone around me was adhering to this.

bonzini 6y ago
The question is whether the review process vets the topic of the paper, the writing, the citations etc., or it's only a screening that avoids revealing company secrets.
pseudalopex 6y ago
What organization are you in? Several people have said AI and Brain don't work that way. Someone from Tech Infra said their papers could get spiked just for not being interesting enough.
- throwaway74453 6y ago
  Brain
sillysaurusx 6y ago
Thank you for speaking up! To be clear, there’s no question it exists. But multiple (admittedly former) google researchers have gone on record saying that this is meant to be a quick scan for trade secrets leakage, and nothing more than that. Certainly not some kind of academic integrity review. That’s what the journal reviewers are for. The point of the scientific process is to be allowed to fail and to be mistaken. And the paper was pretty bland. Sure, she didn’t namedrop all relevant research over the last decade, but why demand a retraction? Especially when the reviewers posted on Reddit that there was a big edit window to make revisions. Adding a few more cites seems like “oh, why not throw in X?” then you both go out to lunch / email an updated version later that day. A retraction is basically “your last few months have been wasted,” right? I’d probably be upset too. Thanks again for the datapoint about the 3 day window. It seems rather selective, but that’s standard for any bigco. I’m just having a hard time stomaching the idea that your ideas need to pass through some anonymous internal review panel where you don’t even know who’s judging you. Is that really what it’s like to be there? Seems strange.
- remote_phone 6y ago
  The fact is that you’re wrong. There is an thorough internal review process. It’s not just a rubber stamp. So please stop spreading misinformation. What Dr. Gebru went through was standard.
- throwaway74453 6y ago
  Typically this is not something you worry about too much. You work on a research project, you talk to people around you, your research manager and everyone around you pretty openly, and if something was going very wrong, you'd likely know because someone would have told you along the way. So by the time you are submitting for internal review, the expectation is that you will almost certainly pass, but typically the reviewers tale their job pretty seriously (and so did I when I served as one) and spent a week or two reading through your paper in their "spare" time and writing a comment on the science, the method, the conclusions etc. The not-leaking-secrets part is certainly there, but that's the lowest bar to pass, so one rarely worries about it practically if one works on public datasets and doing fundamental science. My experience might be different from what Dr Gebru was going through since I never rubbed against anything that could have been considered company secret. My work was entirely academic and I never felt that I was restricted in any way in the questions that I could ask or the papers I could write. That is likely very different when you are criticizing a product, using internal data etc, which might have been the case with her. It also seems that she was in no way diplomatic about her actions. When you do fundamental research there, it's as free or possibly even freer than standard academic institutions. As I said, I personally never felt any implicit let alone explicit forces telling what to work on / what to avoid.

bonzini 6y ago

The question is whether the review process vets the topic of the paper, the writing, the citations etc., or it's only a screening that avoids revealing company secrets.

pseudalopex 6y ago

What organization are you in? Several people have said AI and Brain don't work that way. Someone from Tech Infra said their papers could get spiked just for not being interesting enough.

throwaway74453 6y ago
Brain

sillysaurusx 6y ago

Thank you for speaking up! To be clear, there’s no question it exists. But multiple (admittedly former) google researchers have gone on record saying that this is meant to be a quick scan for trade secrets leakage, and nothing more than that. Certainly not some kind of academic integrity review. That’s what the journal reviewers are for. The point of the scientific process is to be allowed to fail and to be mistaken. And the paper was pretty bland. Sure, she didn’t namedrop all relevant research over the last decade, but why demand a retraction? Especially when the reviewers posted on Reddit that there was a big edit window to make revisions. Adding a few more cites seems like “oh, why not throw in X?” then you both go out to lunch / email an updated version later that day. A retraction is basically “your last few months have been wasted,” right? I’d probably be upset too. Thanks again for the datapoint about the 3 day window. It seems rather selective, but that’s standard for any bigco. I’m just having a hard time stomaching the idea that your ideas need to pass through some anonymous internal review panel where you don’t even know who’s judging you. Is that really what it’s like to be there? Seems strange.

remote_phone 6y ago
The fact is that you’re wrong. There is an thorough internal review process. It’s not just a rubber stamp. So please stop spreading misinformation. What Dr. Gebru went through was standard.
throwaway74453 6y ago
Typically this is not something you worry about too much. You work on a research project, you talk to people around you, your research manager and everyone around you pretty openly, and if something was going very wrong, you'd likely know because someone would have told you along the way. So by the time you are submitting for internal review, the expectation is that you will almost certainly pass, but typically the reviewers tale their job pretty seriously (and so did I when I served as one) and spent a week or two reading through your paper in their "spare" time and writing a comment on the science, the method, the conclusions etc. The not-leaking-secrets part is certainly there, but that's the lowest bar to pass, so one rarely worries about it practically if one works on public datasets and doing fundamental science. My experience might be different from what Dr Gebru was going through since I never rubbed against anything that could have been considered company secret. My work was entirely academic and I never felt that I was restricted in any way in the questions that I could ask or the papers I could write. That is likely very different when you are criticizing a product, using internal data etc, which might have been the case with her. It also seems that she was in no way diplomatic about her actions. When you do fundamental research there, it's as free or possibly even freer than standard academic institutions. As I said, I personally never felt any implicit let alone explicit forces telling what to work on / what to avoid.