5 ms·

What really kills me is that Google claimed previous papers weren't up to their standard, but then ended up citing them in later papers that they've had press e

by tedivm 4y ago

What really kills me is that Google claimed previous papers weren't up to their standard, but then ended up citing them in later papers that they've had press events around. It's really unfortunate that so much of our science is being done at large companies rather than universities, since the companies have a bias towards their bottom line which keeps them from publishing things that might damage that.

godelski 4y ago
Papers may not meet the standards for publication but still be useful. For example, I have a paper with 50 citations that is having difficulties getting published (either "not novel enough" or "not motivating"). There's also a randomness to review, as NeuralIPS has continually demonstrated. I should also mention that you can also pad your citations this way and build your h-index. It is metric hacking, but if it isn't too egregious this is normal. Classic example of Goodhart's law.
muh_gradle 4y ago
Which papers are you referring to?
- dekhn 4y ago
  https://www.techrxiv.org/articles/preprint/The_Carbon_Footprint_of_Machine_Learning_Training_Will_Plateau_Then_Shrink/19139645 https://www.techrxiv.org/articles/preprint/The_Carbon_Footpr... cites Bender, E.M., et al., 2021, On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. ACM Conference on Fairness, Accountability, and Transparency. In short, Jeff Dean (and others at Google) cited the finally published (not by Google) version of the Parrots paper. The citation [20] is in the context of the paper: "The opening quote in Section 1 is based on a 2019 study from the University of Massachusetts (UMass) that estimated the environmental impact of training [2]. More than 1000 papers cite this paper as the source for the impact on carbon emissions of ML models, e.g., [1,6,7,17,18,19,20]." in which the authors of this paper point out that the Parrots paper used what is likely a large overestimate of the carbon emissions from training. This is consistent with my understanding (that the power/emissions part of the Parrots paper isn't well backed by evidence).
  - muh_gradle 4y ago
    Ahh thank you for the context. I was familiar with the Timnit Gebru incident, but I haven't followed up on it since it happened.
pastacacioepepe 4y ago
But...but...there's nothing private investors and the free market can't do! Seriously, who thought that science for profit would net the best results for the collective benefit? You take science but replace truth with profit as the ultimate objective. I'm sure that's going to work well. The day we stop seeking profit in endeavors that can only be pursued properly at a complete economic loss, we're going to solve so many problems at once.
- skoskie 4y ago
  > Seriously, who thought that science for profit would net the best results for the collective benefit? Nobody, and I don’t think “best results” or “collective benefit” were the goal when science started getting more commercialized. The idea that good inventions would benefit all rather than patent holders must drive a true capitalist crazy.
  - pastacacioepepe 4y ago
    Yet even Adam Smith bases his entire theory on the supposed collective benefit of the invisible hand. Without collective benefit, there's nothing positive left to it.
mc32 4y ago
Universities have their own biases as well --whether it's to pad one's CV, the PI's CV, or whatever, but they are also not free from interference against 'pure' research. There is a lot of "garbage" research that happens at universities. Yes, we do expect quite a bit of research to be dead-ends --that's the nature of research, but there is a lot of research that is knowingly fraudulent or uses very suspect data and or methodology.
- skoskie 4y ago
  Out of curiosity, are there any institutions that meets a “pure” research standard? NIST? Or is that standard unattainable because undue influence exists everywhere?
  - canaus 4y ago
    Influence exists everywhere because without influence the studies wouldn't exist in the first place.
    - azinman2 4y ago
      And people need careers, successful publications and grants make that possible. That isn’t to say everything is trash— lots of good stuff is still done with these requirements.
    - lupire 4y ago
      Unless you have a wealthy aristocracy and some of them are bored so they create a lot of the major science of past couple of centuries.
  - guhidalg 4y ago
    Get a private wealthy benefactor that likes your work, then you can avoid undue influence.
    - thebean11 4y ago
      ..until the wealthy benefactor decides to exert it anyway
oh_sigh 4y ago
What papers are you talking about specifically? Were they reworked at all after being told they weren't up to standard? Does Google have a standard where only papers they would publish can be cited? Imagine if Google had a standard that all papers they publish must be in Comic Sans font. Researcher refuses, publishes paper independently, and then someone at Google publishes a paper in Comic Sans referring to that paper. Is that wrong? After all, Google doesn't have a standard that all cited papers must be published in Comic Sans as well.
- joshuamorton 4y ago
  The controversy here is specifically around Stochastic Parrots, by among others Timnit Gebru and "Schmargaret Smitchell". Timnit had recently been fired by Google for a controversy that started when leadership prevented her and Margaret from being coauthors on the paper (which is critical of Google in some ways) and Mitchell was fired a few months later. Google[0] claims the paper didn't meet the bar for publication and ignored too much relevant research. The authors obviously dispute these claims (I generally side with the authors that the "ignored research" line is a stretch). The then amusing thing is that a number of Google authored or coauthored papers still cite it, for various reasons[1][2] (but there's another like half dozen or so Google papers at least in the ~500 citations). So the entire controversy is that the paper wasn't up to academic standards, except that empirically it clearly was. > Were they reworked at all after being told they weren't up to standard? There was an attempt by the authors to respond to the internal criticism, but this wasn't met constructively. [0]: https://docs.google.com/document/d/1f2kYWDXwhzYnq8ebVtuk9CqQqz7ScqxhSIxeYGrWjK0/edit https://docs.google.com/document/d/1f2kYWDXwhzYnq8ebVtuk9CqQ... [1]: https://arxiv.org/pdf/2107.06499.pdf https://arxiv.org/pdf/2107.06499.pdf [2]: https://arxiv.org/pdf/2109.01652.pdf https://arxiv.org/pdf/2109.01652.pdf
  - gwern 4y ago
    > There was an attempt by the authors to respond to the internal criticism, but this wasn't met constructively. Really? Because people claiming to be the reviewers at the time said that the version they reviewed was a lot worse than the final version published much later. (And that version wasn't too great either.)
true_religion 4y ago
Can't you cite things that you disagree with? Or cite background material that you feel is subpar? That would be the norm especially if your research paper is focused on new technology. For example you would write Herbet et co, in Journal Blah found results of X with method Y. However method Y has flaws of A, B, and C... Even if a pervious study was utterly stupid in your opinion, if it's been widely publicized you still need to talk about it or else risk being accused of missing the basics.
- haswell 4y ago
  This was my thought as well. Merely citing the material isn’t necessarily a problem on its own. The nature of the citation and how that impacts the paper doing the citing would need to be examined to form a meaningful opinion about Google’s behavior here.
- lupire 4y ago
  You are right but the implication was that the citations were not rebuttals.
  - true_religion 4y ago
    Then the answer is that even papers that don't meet the "high" standard that every organization and journal likes to think they hold themselves to when publishing, still contain relevant data and explanations that are citable. Although let's be clear... this is bureaucratic issue. Some journals only publish English language papers. No matter how impactful your paper is, they'll never publish it if you wrote it in Tamil. Google only publishes papers that doesn't directly insult them, and spread fear about their own proprietary technology while providing no clear solutions. The paper may be brilliant but they definitely won't put their money and name behind it. Since they (Google) are ideologically opposed to the paper's findings, I can't imagine they are citing it to talk about how brilliant it is though...
DashAnimal 4y ago
Do you have an example of Google citing a "below standard" paper? Curious to see.
buzzthro 4y ago
I mean, for all we know, those papers could have been cited as "look here's how not to do it or here's how we're different/better". There are positive and negative citations in papers.
protastus 4y ago
From an author's perspective, citing a paper is not an endorsement. For example, a paper's intro might have an overview of how others have attempted to solve a problem before. Citations there will comprise a very concise summary of techniques from others, often pointing out limitations. From the perspective of a PR team or agency, they aren't in a position to independently evaluate papers, let alone disavow a paper that carries the name of the company.

by tedivm 4y ago

godelski 4y ago

Papers may not meet the standards for publication but still be useful. For example, I have a paper with 50 citations that is having difficulties getting published (either "not novel enough" or "not motivating"). There's also a randomness to review, as NeuralIPS has continually demonstrated. I should also mention that you can also pad your citations this way and build your h-index. It is metric hacking, but if it isn't too egregious this is normal. Classic example of Goodhart's law.

muh_gradle 4y ago

Which papers are you referring to?

dekhn 4y ago
https://www.techrxiv.org/articles/preprint/The_Carbon_Footprint_of_Machine_Learning_Training_Will_Plateau_Then_Shrink/19139645 https://www.techrxiv.org/articles/preprint/The_Carbon_Footpr... cites Bender, E.M., et al., 2021, On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. ACM Conference on Fairness, Accountability, and Transparency. In short, Jeff Dean (and others at Google) cited the finally published (not by Google) version of the Parrots paper. The citation [20] is in the context of the paper: "The opening quote in Section 1 is based on a 2019 study from the University of Massachusetts (UMass) that estimated the environmental impact of training [2]. More than 1000 papers cite this paper as the source for the impact on carbon emissions of ML models, e.g., [1,6,7,17,18,19,20]." in which the authors of this paper point out that the Parrots paper used what is likely a large overestimate of the carbon emissions from training. This is consistent with my understanding (that the power/emissions part of the Parrots paper isn't well backed by evidence).
- muh_gradle 4y ago
  Ahh thank you for the context. I was familiar with the Timnit Gebru incident, but I haven't followed up on it since it happened.

pastacacioepepe 4y ago

But...but...there's nothing private investors and the free market can't do! Seriously, who thought that science for profit would net the best results for the collective benefit? You take science but replace truth with profit as the ultimate objective. I'm sure that's going to work well. The day we stop seeking profit in endeavors that can only be pursued properly at a complete economic loss, we're going to solve so many problems at once.

skoskie 4y ago
> Seriously, who thought that science for profit would net the best results for the collective benefit? Nobody, and I don’t think “best results” or “collective benefit” were the goal when science started getting more commercialized. The idea that good inventions would benefit all rather than patent holders must drive a true capitalist crazy.
- pastacacioepepe 4y ago
  Yet even Adam Smith bases his entire theory on the supposed collective benefit of the invisible hand. Without collective benefit, there's nothing positive left to it.

mc32 4y ago

Universities have their own biases as well --whether it's to pad one's CV, the PI's CV, or whatever, but they are also not free from interference against 'pure' research. There is a lot of "garbage" research that happens at universities. Yes, we do expect quite a bit of research to be dead-ends --that's the nature of research, but there is a lot of research that is knowingly fraudulent or uses very suspect data and or methodology.

skoskie 4y ago
Out of curiosity, are there any institutions that meets a “pure” research standard? NIST? Or is that standard unattainable because undue influence exists everywhere?
- canaus 4y ago
  Influence exists everywhere because without influence the studies wouldn't exist in the first place.
  - azinman2 4y ago
    And people need careers, successful publications and grants make that possible. That isn’t to say everything is trash— lots of good stuff is still done with these requirements.
  - lupire 4y ago
    Unless you have a wealthy aristocracy and some of them are bored so they create a lot of the major science of past couple of centuries.
- guhidalg 4y ago
  Get a private wealthy benefactor that likes your work, then you can avoid undue influence.
  - thebean11 4y ago
    ..until the wealthy benefactor decides to exert it anyway

oh_sigh 4y ago

What papers are you talking about specifically? Were they reworked at all after being told they weren't up to standard? Does Google have a standard where only papers they would publish can be cited? Imagine if Google had a standard that all papers they publish must be in Comic Sans font. Researcher refuses, publishes paper independently, and then someone at Google publishes a paper in Comic Sans referring to that paper. Is that wrong? After all, Google doesn't have a standard that all cited papers must be published in Comic Sans as well.

joshuamorton 4y ago
The controversy here is specifically around Stochastic Parrots, by among others Timnit Gebru and "Schmargaret Smitchell". Timnit had recently been fired by Google for a controversy that started when leadership prevented her and Margaret from being coauthors on the paper (which is critical of Google in some ways) and Mitchell was fired a few months later. Google[0] claims the paper didn't meet the bar for publication and ignored too much relevant research. The authors obviously dispute these claims (I generally side with the authors that the "ignored research" line is a stretch). The then amusing thing is that a number of Google authored or coauthored papers still cite it, for various reasons[1][2] (but there's another like half dozen or so Google papers at least in the ~500 citations). So the entire controversy is that the paper wasn't up to academic standards, except that empirically it clearly was. > Were they reworked at all after being told they weren't up to standard? There was an attempt by the authors to respond to the internal criticism, but this wasn't met constructively. [0]: https://docs.google.com/document/d/1f2kYWDXwhzYnq8ebVtuk9CqQqz7ScqxhSIxeYGrWjK0/edit https://docs.google.com/document/d/1f2kYWDXwhzYnq8ebVtuk9CqQ... [1]: https://arxiv.org/pdf/2107.06499.pdf https://arxiv.org/pdf/2107.06499.pdf [2]: https://arxiv.org/pdf/2109.01652.pdf https://arxiv.org/pdf/2109.01652.pdf
- gwern 4y ago
  > There was an attempt by the authors to respond to the internal criticism, but this wasn't met constructively. Really? Because people claiming to be the reviewers at the time said that the version they reviewed was a lot worse than the final version published much later. (And that version wasn't too great either.)

true_religion 4y ago

Can't you cite things that you disagree with? Or cite background material that you feel is subpar? That would be the norm especially if your research paper is focused on new technology. For example you would write Herbet et co, in Journal Blah found results of X with method Y. However method Y has flaws of A, B, and C... Even if a pervious study was utterly stupid in your opinion, if it's been widely publicized you still need to talk about it or else risk being accused of missing the basics.

haswell 4y ago
This was my thought as well. Merely citing the material isn’t necessarily a problem on its own. The nature of the citation and how that impacts the paper doing the citing would need to be examined to form a meaningful opinion about Google’s behavior here.
lupire 4y ago
You are right but the implication was that the citations were not rebuttals.
- true_religion 4y ago
  Then the answer is that even papers that don't meet the "high" standard that every organization and journal likes to think they hold themselves to when publishing, still contain relevant data and explanations that are citable. Although let's be clear... this is bureaucratic issue. Some journals only publish English language papers. No matter how impactful your paper is, they'll never publish it if you wrote it in Tamil. Google only publishes papers that doesn't directly insult them, and spread fear about their own proprietary technology while providing no clear solutions. The paper may be brilliant but they definitely won't put their money and name behind it. Since they (Google) are ideologically opposed to the paper's findings, I can't imagine they are citing it to talk about how brilliant it is though...

DashAnimal 4y ago

Do you have an example of Google citing a "below standard" paper? Curious to see.

buzzthro 4y ago

I mean, for all we know, those papers could have been cited as "look here's how not to do it or here's how we're different/better". There are positive and negative citations in papers.

protastus 4y ago

From an author's perspective, citing a paper is not an endorsement. For example, a paper's intro might have an overview of how others have attempted to solve a problem before. Citations there will comprise a very concise summary of techniques from others, often pointing out limitations. From the perspective of a PR team or agency, they aren't in a position to independently evaluate papers, let alone disavow a paper that carries the name of the company.