14 ms·

And the examples of issues flagged in review that Jeff keeps highlighting—like Timnit’s alleged failure to mention recent work to reduce the environmental impac

And the examples of issues flagged in review that Jeff keeps highlighting—like Timnit’s alleged failure to mention recent work to reduce the environmental impact of large models—are themselves a bit worrisome. Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks.

omgwtfbyobbq 6y ago
That was my impression as well. Having said that, if Jeff were to make public the paper, criticisms of the paper, and improvements made to address the problems described in the paper, that could go a long way towards clearing the air.
- gcr 6y ago
  oh yes absolutely, i would love for timmit or jeff to release this paper as-is so we can see what happened. could be a good read.
  - UncleMeat 6y ago
    The draft is rolling around the web already.
    - treis 6y ago
      link?
      - ninjin 6y ago
        A second hand account from a VentureBeat journalist is the best I could find [1]. As a researcher with more than a decade of experience in Natural Language Processing, what is described by the journalist in regards to the paper content seems to be non-controversial and nothing out of the ordinary from this kind of work. If anyone could find an actual leak I would be more than excited to have a look at it using my own eyes rather than filtered through someone else’s. [1]: https://venturebeat.com/2020/12/03/ai-ethics-pioneers-exit-from-google-involved-research-into-risks-and-inequality-in-large-language-models https://venturebeat.com/2020/12/03/ai-ethics-pioneers-exit-f...
- htrp 6y ago
  Abstract from a reviewer (source reddit /r/ml) Abstract The past three years of work in natural language processing have been characterized by the development and deployment of ever larger language models, especially for English. GPT-2, GPT-3, BERT and its variants have pushed the boundaries of the possible both through architectural innovations and through sheer size. Using these pre- trained models and the methodology of fine-tuning them for specific tasks, researchers have extended the state of the art on a wide array of tasks as measured by leaderboards on specific benchmarks for English. In this paper, we take a step back and ask: How big is too big? What are the possible risks associated with this technology and what paths are available for mitigating those risks? We end with recommendations including weighing the environmental and financial costs first, investing resources into curating and carefully documenting datasets rather than ingesting everything on the web, carrying out pre-development exercises evaluating how the planned approach fits into research and development goals and supports stakeholder values, and encouraging research directions beyond ever larger language models. Anyone have the actual paper?
  - ninjin 6y ago
    Link to the Reddit thread: https://old.reddit.com/r/MachineLearning/comments/k69eq0/n_the_abstract_of_the_paper_that_led_to_timnit/ https://old.reddit.com/r/MachineLearning/comments/k69eq0/n_t... I have plenty of experience in Natural Language Processing (NLP), but I am not an expert in ethics and bias – although I have read my fair share of papers on it. To me, the abstract comes across as modest, very reasonable, and exploring questions highly relevant to the community as a whole. Sure, if I was to review it I would be picky in regards to any strong empirical claims since I suspect it would be difficult to demonstrate conclusively some aspects of what they hint towards in the abstract. But as a position paper it looks better than plenty of work already published at top-tier NLP venues and I doubt that it could not get accepted on academic merits. Still, to echo the parent, does anyone have the paper in its entirety?
acoard 6y ago
Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. They couch these in terms of rigour, but the substance of the problem has to do with her criticisms against AI. Ultimately it makes the whole Ethical AI department look more like a rubber stamp for Google.
- orsenthil 6y ago
  > Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. I didn't read that. I read the person _demanded_ who said a particular critical feedback, or questioned the approaches instead of addressing them. The person gave the ultimatum to resign if details were not shared.
  - sudosysgen 6y ago
    Unless she lied in her first e-mail, which it doesn't seem like she did, the reason she made those demands is because they asked for a retraction of the paper without indicating why the paper should be retracted. Asking for the identity of people that have the authority to ask for a withdrawal of your research without stating their issues with it seems understandable, if excessive. But maybe I misunderstood something.
    - lokar 6y ago
      AIUI, they asked her to retract it because she submitted it before getting final approval, and then they in fact decided not to approve it.
      - Footkerchief 6y ago
        Dean's statement is clear that it was approved before being submitted: > Unfortunately, this particular paper was only shared with a day’s notice before its deadline — we require two weeks for this sort of review — and then instead of awaiting reviewer feedback, it was approved for submission and submitted. A cross functional team then reviewed the paper as part of our regular process and the authors were informed that it didn’t meet our bar for publication and were given feedback about why. [...] We acknowledge that the authors were extremely disappointed with the decision that Megan and I ultimately made, especially as they’d already submitted the paper. There is no statement at all of how to reconcile "approved for submission" with "didn’t meet our bar for publication", which probably means that there is no reconciliation, and the cancellation was done outside normal process.
that_guy_iain 6y ago
> Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks. I get the impression that she wrote a hit piece on Google and published with Google's name. For me, it's correct they demand a retraction. It's simply unprofessional to critque your company for something while not mentioning the work they're doing to combat that.
- chejazi 6y ago
  > I get the impression that she wrote a hit piece Hopefully the paper gets leaked so we can judge for ourselves.
  - tmathmeyer 6y ago
    I've read it internally. It's rather bland and doesn't actually indite google. I'm inclined to believe that it was a paper-submission-process issue.
    - Ar-Curunir 6y ago
      That disagrees with what Dean himself is saying...
- freeone3000 6y ago
  Company with unethical practices hires ethics researcher Ethics researcher publishes piece critical of company's activities Company is shocked as to how this could happen.
  - acituan 6y ago
    Ethics researcher omits relevant research in paper to smuggle in a particular agenda. Company calls it out. Researcher realizes the limits of her narrative setting powers. Non-vocal majority is happy to see runaway activism being curbed in corporate settings. Slow march through institutions is slowed down for a day.
trhway 6y ago
Google isnt publicly funded academic institution. Whatever they are doing, in particular publishing, is part of the business/PR. So if the management sees something not good for business it is a reasonable that they decided to not do it. If i were a shareholder i can see how i may have questioned why a person being paid $1M+/year (my understanding this is minimum what a manager in AI at Google would be making) for publicly disparaging Google. Even more, it sounds like Google didn't ask originally for retraction, they just asked to take into account the newer research contradicting the paper - the thing that any researcher valuing integrity over agenda wouldn't refuse. If somebody wants to do that research and publishing they just have to find another source of funding, i guess. Anyway, the firing wasn't over the paper, the firing was over the unacceptably unprofessional reaction to it.
- albatruss 6y ago
  From a certain narrow, selfish perspective it's reasonable for Google to not want to have an AI ethics department placing a check on their leading edge research at all. Fortunately, we don't live in a world where corporations are the ones to determine right from wrong with total impunity.
  - trhway 6y ago
    > AI ethics department placing a check on their leading edge research that reminds how in USSR each non-miniscule factory, organization, etc. had "the department #1" - it was an ideological check and control department which at sufficiently large/important organizations even included KGB officers.
    - albatruss 6y ago
      You have identified a similarity between two situations, but it is not a similarity that matters. The distinction that matters is one of normativity, and on that measure there is clearly no equivalence to be drawn here.
      - trhway 6y ago
visarga 6y ago
> Jeff gives the impression that they demanded retraction (!) In paper reviews you can often see reviewers asking the authors to rewrite, clarify, add extra experiments, add missing citations. It's all normal.
- UncleMeat 6y ago
  Usually those are "accept with minor revisions" or "revise and resubmit". Rarely are they grounds for complete rejection. This is extra true for internal review, since the actual conference review process would provide an additional layer to ensure that the scholarship was strong.
jariel 6y ago
"Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks." Representing a more truthful reality is not 'softening'. It's only 'softening' for those who have an already accepted, extremist view, and for whom any evidence to the contrary doesn't help their arguments. While initially sympathetic to the author - the more I read - the more I have completely the opposite view.
tylerl 6y ago
The communication doesn't give that impression; instead it says that the paper makes claims that ignore significant and credible challenges to those claims. Dean said that these factors would need to be addressed, not agreed with. Publishing a transparently one-sided paper in Google's name would be a problem, not because of the side it picks, but because it suggests the researchers are too ideologically motivated to see the see the problem clearly. Ironically, it indicates systemic bias on the part of the researchers who are explicitly trying to eliminate systemic bias. That's just a bit too relevant to ignore.
- GVIrish 6y ago
  If that is indeed why the demand for retraction, why didn't they state that up front in the meeting where they told Timnit she needed to retract the paper or remove her name? Instead they initially refused to tell her the reasons for the demand for retraction. They didn't give her a chance to address those factors at first. Later they had a manager read the confidential feedback on the paper in question, but still didn't leg her read it herself. If that feedback was only saying that the paper lacked relevant new context and advancements, why were they being so cagey about it? Something doesn't smell right about that.

omgwtfbyobbq 6y ago

That was my impression as well. Having said that, if Jeff were to make public the paper, criticisms of the paper, and improvements made to address the problems described in the paper, that could go a long way towards clearing the air.

gcr 6y ago
oh yes absolutely, i would love for timmit or jeff to release this paper as-is so we can see what happened. could be a good read.
- UncleMeat 6y ago
  The draft is rolling around the web already.
  - treis 6y ago
    link?
    - ninjin 6y ago
      A second hand account from a VentureBeat journalist is the best I could find [1]. As a researcher with more than a decade of experience in Natural Language Processing, what is described by the journalist in regards to the paper content seems to be non-controversial and nothing out of the ordinary from this kind of work. If anyone could find an actual leak I would be more than excited to have a look at it using my own eyes rather than filtered through someone else’s. [1]: https://venturebeat.com/2020/12/03/ai-ethics-pioneers-exit-from-google-involved-research-into-risks-and-inequality-in-large-language-models https://venturebeat.com/2020/12/03/ai-ethics-pioneers-exit-f...
htrp 6y ago
Abstract from a reviewer (source reddit /r/ml) Abstract The past three years of work in natural language processing have been characterized by the development and deployment of ever larger language models, especially for English. GPT-2, GPT-3, BERT and its variants have pushed the boundaries of the possible both through architectural innovations and through sheer size. Using these pre- trained models and the methodology of fine-tuning them for specific tasks, researchers have extended the state of the art on a wide array of tasks as measured by leaderboards on specific benchmarks for English. In this paper, we take a step back and ask: How big is too big? What are the possible risks associated with this technology and what paths are available for mitigating those risks? We end with recommendations including weighing the environmental and financial costs first, investing resources into curating and carefully documenting datasets rather than ingesting everything on the web, carrying out pre-development exercises evaluating how the planned approach fits into research and development goals and supports stakeholder values, and encouraging research directions beyond ever larger language models. Anyone have the actual paper?
- ninjin 6y ago
  Link to the Reddit thread: https://old.reddit.com/r/MachineLearning/comments/k69eq0/n_the_abstract_of_the_paper_that_led_to_timnit/ https://old.reddit.com/r/MachineLearning/comments/k69eq0/n_t... I have plenty of experience in Natural Language Processing (NLP), but I am not an expert in ethics and bias – although I have read my fair share of papers on it. To me, the abstract comes across as modest, very reasonable, and exploring questions highly relevant to the community as a whole. Sure, if I was to review it I would be picky in regards to any strong empirical claims since I suspect it would be difficult to demonstrate conclusively some aspects of what they hint towards in the abstract. But as a position paper it looks better than plenty of work already published at top-tier NLP venues and I doubt that it could not get accepted on academic merits. Still, to echo the parent, does anyone have the paper in its entirety?

acoard 6y ago

Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. They couch these in terms of rigour, but the substance of the problem has to do with her criticisms against AI. Ultimately it makes the whole Ethical AI department look more like a rubber stamp for Google.

orsenthil 6y ago
> Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. I didn't read that. I read the person _demanded_ who said a particular critical feedback, or questioned the approaches instead of addressing them. The person gave the ultimatum to resign if details were not shared.
- sudosysgen 6y ago
  Unless she lied in her first e-mail, which it doesn't seem like she did, the reason she made those demands is because they asked for a retraction of the paper without indicating why the paper should be retracted. Asking for the identity of people that have the authority to ask for a withdrawal of your research without stating their issues with it seems understandable, if excessive. But maybe I misunderstood something.
  - lokar 6y ago
    AIUI, they asked her to retract it because she submitted it before getting final approval, and then they in fact decided not to approve it.
    - Footkerchief 6y ago
      Dean's statement is clear that it was approved before being submitted: > Unfortunately, this particular paper was only shared with a day’s notice before its deadline — we require two weeks for this sort of review — and then instead of awaiting reviewer feedback, it was approved for submission and submitted. A cross functional team then reviewed the paper as part of our regular process and the authors were informed that it didn’t meet our bar for publication and were given feedback about why. [...] We acknowledge that the authors were extremely disappointed with the decision that Megan and I ultimately made, especially as they’d already submitted the paper. There is no statement at all of how to reconcile "approved for submission" with "didn’t meet our bar for publication", which probably means that there is no reconciliation, and the cancellation was done outside normal process.

that_guy_iain 6y ago

> Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks. I get the impression that she wrote a hit piece on Google and published with Google's name. For me, it's correct they demand a retraction. It's simply unprofessional to critque your company for something while not mentioning the work they're doing to combat that.

chejazi 6y ago
> I get the impression that she wrote a hit piece Hopefully the paper gets leaked so we can judge for ourselves.
- tmathmeyer 6y ago
  I've read it internally. It's rather bland and doesn't actually indite google. I'm inclined to believe that it was a paper-submission-process issue.
  - Ar-Curunir 6y ago
    That disagrees with what Dean himself is saying...
freeone3000 6y ago
Company with unethical practices hires ethics researcher Ethics researcher publishes piece critical of company's activities Company is shocked as to how this could happen.
- acituan 6y ago
  Ethics researcher omits relevant research in paper to smuggle in a particular agenda. Company calls it out. Researcher realizes the limits of her narrative setting powers. Non-vocal majority is happy to see runaway activism being curbed in corporate settings. Slow march through institutions is slowed down for a day.

trhway 6y ago

Google isnt publicly funded academic institution. Whatever they are doing, in particular publishing, is part of the business/PR. So if the management sees something not good for business it is a reasonable that they decided to not do it. If i were a shareholder i can see how i may have questioned why a person being paid $1M+/year (my understanding this is minimum what a manager in AI at Google would be making) for publicly disparaging Google. Even more, it sounds like Google didn't ask originally for retraction, they just asked to take into account the newer research contradicting the paper - the thing that any researcher valuing integrity over agenda wouldn't refuse. If somebody wants to do that research and publishing they just have to find another source of funding, i guess. Anyway, the firing wasn't over the paper, the firing was over the unacceptably unprofessional reaction to it.

albatruss 6y ago
From a certain narrow, selfish perspective it's reasonable for Google to not want to have an AI ethics department placing a check on their leading edge research at all. Fortunately, we don't live in a world where corporations are the ones to determine right from wrong with total impunity.
- trhway 6y ago
  > AI ethics department placing a check on their leading edge research that reminds how in USSR each non-miniscule factory, organization, etc. had "the department #1" - it was an ideological check and control department which at sufficiently large/important organizations even included KGB officers.
  - albatruss 6y ago
    You have identified a similarity between two situations, but it is not a similarity that matters. The distinction that matters is one of normativity, and on that measure there is clearly no equivalence to be drawn here.
    - trhway 6y ago

visarga 6y ago

> Jeff gives the impression that they demanded retraction (!) In paper reviews you can often see reviewers asking the authors to rewrite, clarify, add extra experiments, add missing citations. It's all normal.

UncleMeat 6y ago
Usually those are "accept with minor revisions" or "revise and resubmit". Rarely are they grounds for complete rejection. This is extra true for internal review, since the actual conference review process would provide an additional layer to ensure that the scholarship was strong.

jariel 6y ago

"Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks." Representing a more truthful reality is not 'softening'. It's only 'softening' for those who have an already accepted, extremist view, and for whom any evidence to the contrary doesn't help their arguments. While initially sympathetic to the author - the more I read - the more I have completely the opposite view.

tylerl 6y ago

The communication doesn't give that impression; instead it says that the paper makes claims that ignore significant and credible challenges to those claims. Dean said that these factors would need to be addressed, not agreed with. Publishing a transparently one-sided paper in Google's name would be a problem, not because of the side it picks, but because it suggests the researchers are too ideologically motivated to see the see the problem clearly. Ironically, it indicates systemic bias on the part of the researchers who are explicitly trying to eliminate systemic bias. That's just a bit too relevant to ignore.

GVIrish 6y ago
If that is indeed why the demand for retraction, why didn't they state that up front in the meeting where they told Timnit she needed to retract the paper or remove her name? Instead they initially refused to tell her the reasons for the demand for retraction. They didn't give her a chance to address those factors at first. Later they had a manager read the confidential feedback on the paper in question, but still didn't leg her read it herself. If that feedback was only saying that the paper lacked relevant new context and advancements, why were they being so cagey about it? Something doesn't smell right about that.