5 ms·

That was my impression as well. Having said that, if Jeff were to make public the paper, criticisms of the paper, and improvements made to address the problems

That was my impression as well. Having said that, if Jeff were to make public the paper, criticisms of the paper, and improvements made to address the problems described in the paper, that could go a long way towards clearing the air.

gcr 6y ago
oh yes absolutely, i would love for timmit or jeff to release this paper as-is so we can see what happened. could be a good read.
- UncleMeat 6y ago
  The draft is rolling around the web already.
  - treis 6y ago
    link?
    - ninjin 6y ago
      A second hand account from a VentureBeat journalist is the best I could find [1]. As a researcher with more than a decade of experience in Natural Language Processing, what is described by the journalist in regards to the paper content seems to be non-controversial and nothing out of the ordinary from this kind of work. If anyone could find an actual leak I would be more than excited to have a look at it using my own eyes rather than filtered through someone else’s. [1]: https://venturebeat.com/2020/12/03/ai-ethics-pioneers-exit-from-google-involved-research-into-risks-and-inequality-in-large-language-models https://venturebeat.com/2020/12/03/ai-ethics-pioneers-exit-f...
htrp 6y ago
Abstract from a reviewer (source reddit /r/ml) Abstract The past three years of work in natural language processing have been characterized by the development and deployment of ever larger language models, especially for English. GPT-2, GPT-3, BERT and its variants have pushed the boundaries of the possible both through architectural innovations and through sheer size. Using these pre- trained models and the methodology of fine-tuning them for specific tasks, researchers have extended the state of the art on a wide array of tasks as measured by leaderboards on specific benchmarks for English. In this paper, we take a step back and ask: How big is too big? What are the possible risks associated with this technology and what paths are available for mitigating those risks? We end with recommendations including weighing the environmental and financial costs first, investing resources into curating and carefully documenting datasets rather than ingesting everything on the web, carrying out pre-development exercises evaluating how the planned approach fits into research and development goals and supports stakeholder values, and encouraging research directions beyond ever larger language models. Anyone have the actual paper?
- ninjin 6y ago
  Link to the Reddit thread: https://old.reddit.com/r/MachineLearning/comments/k69eq0/n_the_abstract_of_the_paper_that_led_to_timnit/ https://old.reddit.com/r/MachineLearning/comments/k69eq0/n_t... I have plenty of experience in Natural Language Processing (NLP), but I am not an expert in ethics and bias – although I have read my fair share of papers on it. To me, the abstract comes across as modest, very reasonable, and exploring questions highly relevant to the community as a whole. Sure, if I was to review it I would be picky in regards to any strong empirical claims since I suspect it would be difficult to demonstrate conclusively some aspects of what they hint towards in the abstract. But as a position paper it looks better than plenty of work already published at top-tier NLP venues and I doubt that it could not get accepted on academic merits. Still, to echo the parent, does anyone have the paper in its entirety?

gcr 6y ago

oh yes absolutely, i would love for timmit or jeff to release this paper as-is so we can see what happened. could be a good read.

UncleMeat 6y ago
The draft is rolling around the web already.
- treis 6y ago
  link?
  - ninjin 6y ago
    A second hand account from a VentureBeat journalist is the best I could find [1]. As a researcher with more than a decade of experience in Natural Language Processing, what is described by the journalist in regards to the paper content seems to be non-controversial and nothing out of the ordinary from this kind of work. If anyone could find an actual leak I would be more than excited to have a look at it using my own eyes rather than filtered through someone else’s. [1]: https://venturebeat.com/2020/12/03/ai-ethics-pioneers-exit-from-google-involved-research-into-risks-and-inequality-in-large-language-models https://venturebeat.com/2020/12/03/ai-ethics-pioneers-exit-f...

htrp 6y ago

Abstract from a reviewer (source reddit /r/ml) Abstract The past three years of work in natural language processing have been characterized by the development and deployment of ever larger language models, especially for English. GPT-2, GPT-3, BERT and its variants have pushed the boundaries of the possible both through architectural innovations and through sheer size. Using these pre- trained models and the methodology of fine-tuning them for specific tasks, researchers have extended the state of the art on a wide array of tasks as measured by leaderboards on specific benchmarks for English. In this paper, we take a step back and ask: How big is too big? What are the possible risks associated with this technology and what paths are available for mitigating those risks? We end with recommendations including weighing the environmental and financial costs first, investing resources into curating and carefully documenting datasets rather than ingesting everything on the web, carrying out pre-development exercises evaluating how the planned approach fits into research and development goals and supports stakeholder values, and encouraging research directions beyond ever larger language models. Anyone have the actual paper?

ninjin 6y ago
Link to the Reddit thread: https://old.reddit.com/r/MachineLearning/comments/k69eq0/n_the_abstract_of_the_paper_that_led_to_timnit/ https://old.reddit.com/r/MachineLearning/comments/k69eq0/n_t... I have plenty of experience in Natural Language Processing (NLP), but I am not an expert in ethics and bias – although I have read my fair share of papers on it. To me, the abstract comes across as modest, very reasonable, and exploring questions highly relevant to the community as a whole. Sure, if I was to review it I would be picky in regards to any strong empirical claims since I suspect it would be difficult to demonstrate conclusively some aspects of what they hint towards in the abstract. But as a position paper it looks better than plenty of work already published at top-tier NLP venues and I doubt that it could not get accepted on academic merits. Still, to echo the parent, does anyone have the paper in its entirety?