4 ms·

I mentioned this to another commenter as well: You might want to reconsider your stance on emergent abilities in LLMs considering the NeurIPS 2023 best paper w

I mentioned this to another commenter as well: You might want to reconsider your stance on emergent abilities in LLMs considering the NeurIPS 2023 best paper winner is titled: "Are Emergent Abilities of Large Language Models a Mirage?" https://arxiv.org/abs/2304.15004 https://arxiv.org/abs/2304.15004 https://blog.neurips.cc/2023/12/11/announcing-the-neurips-2023-paper-awards/ https://blog.neurips.cc/2023/12/11/announcing-the-neurips-20...

Der_Einzige 3y ago
Papers which get accepted with honors are not necessarily more truthful than papers which have been rejected. Yann LeCunn goes on twitter like any other grad student around NeurIPS or ICML/ICMR and bitterly complains when one of his (many) papers is rejected. Whose more likely to be correct here? Yann LeCunn (the TOP nlp scholar in our field by citations, who does claim that most emergent capabilities are real in other papers), or a NeurIPS best paper winner? My bet is on Yann. Also, consider that some work gets a lot of positivity not for the work itself, but for the people who wrote it. Timnit Gebaru's work was effectively ignored until she got famous for her spat with jeff dean at google. Her citations have exploded as a result, and I don't think that most in the field think that the "stochastic parrot" paper was especially good, and certainly not her other papers which include significant amounts of work dedicated to claiming that LLM training is really bad for the environment (despite a single jet taking AI researchers to conferences being worse for the environment than LLM training circa that paper being written was taking). Doesn't matter that the paper was wrong, it's now highly cited because you get brownie points for citing her work in grievance studies influenced subfields of AI.
- peteradio 3y ago
  Yann LeCun through Meta is incentivized towards maximizing capital return based on local maxima. That is how all business works, there is not really a direct incentive to pushing boundaries beyond what can be immediately monetized.
- nsagent 3y ago
  Please at least read the paper before appealing to authority. It is a well designed set of experiments that clearly demonstrates that the notion of a "phase change" (rapid shift in capabilities) as a popularized by many people claiming emergence is actually a gradual improvement with more data. But if you do want to appeal to Lecun as an authority, then maybe you'll accept that these (re)tweets that clearly indicate he finds the insights from the paper to be valid: https://nitter.1d4.us/ylecun/status/1736479356917063847 https://nitter.1d4.us/ylecun/status/1736479356917063847 https://nitter.1d4.us/rao2z/status/1736464000836309259 https://nitter.1d4.us/rao2z/status/1736464000836309259 (retweeted) As for Timnit, I think you have your timeline confused. Model cards are what put her on the map for most general NLP researchers, which predates her difficulties at Google. 2018: Model cards paper was put on arXiv https://arxiv.org/abs/1810.03993 https://arxiv.org/abs/1810.03993 2019: Major ML organizations start using model cards https://github.com/openai/gpt-2/blob/master/model_card.md https://github.com/openai/gpt-2/blob/master/model_card.md 2020: Model cards become fairly standard https://blog.research.google/2020/07/introducing-model-card-toolkit-for.html https://blog.research.google/2020/07/introducing-model-card-... Dec 2020: Timnit is let go from the ethics team at Google https://www.bbc.com/news/technology-55187611 https://www.bbc.com/news/technology-55187611 EDIT:formatting
- visarga 3y ago
  Gebru and her "Stochastic Parrots" did a big disservice to AI safety turning the debate into a shit-show of identity politics. Now she has her own institute, it was a move up for her career. Her twitter spats with Yann LeCun were legendary. Literally sent him to educate himself and refused to debate him.

Der_Einzige 3y ago

Papers which get accepted with honors are not necessarily more truthful than papers which have been rejected. Yann LeCunn goes on twitter like any other grad student around NeurIPS or ICML/ICMR and bitterly complains when one of his (many) papers is rejected. Whose more likely to be correct here? Yann LeCunn (the TOP nlp scholar in our field by citations, who does claim that most emergent capabilities are real in other papers), or a NeurIPS best paper winner? My bet is on Yann. Also, consider that some work gets a lot of positivity not for the work itself, but for the people who wrote it. Timnit Gebaru's work was effectively ignored until she got famous for her spat with jeff dean at google. Her citations have exploded as a result, and I don't think that most in the field think that the "stochastic parrot" paper was especially good, and certainly not her other papers which include significant amounts of work dedicated to claiming that LLM training is really bad for the environment (despite a single jet taking AI researchers to conferences being worse for the environment than LLM training circa that paper being written was taking). Doesn't matter that the paper was wrong, it's now highly cited because you get brownie points for citing her work in grievance studies influenced subfields of AI.

peteradio 3y ago
Yann LeCun through Meta is incentivized towards maximizing capital return based on local maxima. That is how all business works, there is not really a direct incentive to pushing boundaries beyond what can be immediately monetized.
nsagent 3y ago
Please at least read the paper before appealing to authority. It is a well designed set of experiments that clearly demonstrates that the notion of a "phase change" (rapid shift in capabilities) as a popularized by many people claiming emergence is actually a gradual improvement with more data. But if you do want to appeal to Lecun as an authority, then maybe you'll accept that these (re)tweets that clearly indicate he finds the insights from the paper to be valid: https://nitter.1d4.us/ylecun/status/1736479356917063847 https://nitter.1d4.us/ylecun/status/1736479356917063847 https://nitter.1d4.us/rao2z/status/1736464000836309259 https://nitter.1d4.us/rao2z/status/1736464000836309259 (retweeted) As for Timnit, I think you have your timeline confused. Model cards are what put her on the map for most general NLP researchers, which predates her difficulties at Google. 2018: Model cards paper was put on arXiv https://arxiv.org/abs/1810.03993 https://arxiv.org/abs/1810.03993 2019: Major ML organizations start using model cards https://github.com/openai/gpt-2/blob/master/model_card.md https://github.com/openai/gpt-2/blob/master/model_card.md 2020: Model cards become fairly standard https://blog.research.google/2020/07/introducing-model-card-toolkit-for.html https://blog.research.google/2020/07/introducing-model-card-... Dec 2020: Timnit is let go from the ethics team at Google https://www.bbc.com/news/technology-55187611 https://www.bbc.com/news/technology-55187611 EDIT:formatting
visarga 3y ago
Gebru and her "Stochastic Parrots" did a big disservice to AI safety turning the debate into a shit-show of identity politics. Now she has her own institute, it was a move up for her career. Her twitter spats with Yann LeCun were legendary. Literally sent him to educate himself and refused to debate him.