6 ms·

Almost everything interesting about AI so far has been unexpected emergent behavior, and huge gains through minor insights. While I don't doubt that the current

Almost everything interesting about AI so far has been unexpected emergent behavior, and huge gains through minor insights. While I don't doubt that the current architecture is likely to have a current ceiling below that of peak human intelligence in certain dimensions, it's already surpassed it in some, and there are still gains to be made in others through things like synthetic data. I also don't understand the claims that it doesn't generalize. I currently use it to solve problems that I can absolutely guarantee were not in its training set, and it generalizes well enough. I also think that one of the easiest ways to get it to generalize better would simply be through giving it synthetic data which demonstrates the process of generalizing. It also seems foolish to extrapolate on what we have under the assumption that there won't be key insights/changes in architecture as we get to the limitations of synthetic data wins/multi-modal wins.

jahnu 3y ago
> problems that I can absolutely guarantee were not in its training set Can you share the strongest example?
- Jabrov 3y ago
  Pretty much any coding problem in a unique or private codebase
  - Der_Einzige 3y ago
    There is a difference between interpolation, which the majority of humans are performing daily with coding in private codebases, and genuine extrapolation, which is difficult to prove and difficult to find in high dimensional spaces. LLMs may not be able to easily extrapolate (and when it does it's due to high temperature), but they can interpolate extremely well, and most human growth and innovation today comes from novel interpolations, which are what LLMs are excellent at.
  - jahnu 3y ago
    I asked for the strongest example the OP can share in order to evaluate their claim. If it's so obvious to the OP that generalisation is happening then it should be easy to provide a strong example, right?
  - Xelynega 3y ago
    When I talk to people that use copilot to improve their coding workflow, what I often hear is that copilot can replace boilerplate but not any business logic specific to the problem being solved. It sounds like you have a different experience(copilot is useful for business logic). Do you have maybe any examples of what you mean by "any coding problems"? Is it similar to what I've heard previously(copilot can replace boilerplate) or have LLMs actually solved business problems for you in code?
nsagent 3y ago
I mentioned this to another commenter as well: You might want to reconsider your stance on emergent abilities in LLMs considering the NeurIPS 2023 best paper winner is titled: "Are Emergent Abilities of Large Language Models a Mirage?" https://arxiv.org/abs/2304.15004 https://arxiv.org/abs/2304.15004 https://blog.neurips.cc/2023/12/11/announcing-the-neurips-2023-paper-awards/ https://blog.neurips.cc/2023/12/11/announcing-the-neurips-20...
- Der_Einzige 3y ago
  Papers which get accepted with honors are not necessarily more truthful than papers which have been rejected. Yann LeCunn goes on twitter like any other grad student around NeurIPS or ICML/ICMR and bitterly complains when one of his (many) papers is rejected. Whose more likely to be correct here? Yann LeCunn (the TOP nlp scholar in our field by citations, who does claim that most emergent capabilities are real in other papers), or a NeurIPS best paper winner? My bet is on Yann. Also, consider that some work gets a lot of positivity not for the work itself, but for the people who wrote it. Timnit Gebaru's work was effectively ignored until she got famous for her spat with jeff dean at google. Her citations have exploded as a result, and I don't think that most in the field think that the "stochastic parrot" paper was especially good, and certainly not her other papers which include significant amounts of work dedicated to claiming that LLM training is really bad for the environment (despite a single jet taking AI researchers to conferences being worse for the environment than LLM training circa that paper being written was taking). Doesn't matter that the paper was wrong, it's now highly cited because you get brownie points for citing her work in grievance studies influenced subfields of AI.
  - peteradio 3y ago
    Yann LeCun through Meta is incentivized towards maximizing capital return based on local maxima. That is how all business works, there is not really a direct incentive to pushing boundaries beyond what can be immediately monetized.
  - nsagent 3y ago
    Please at least read the paper before appealing to authority. It is a well designed set of experiments that clearly demonstrates that the notion of a "phase change" (rapid shift in capabilities) as a popularized by many people claiming emergence is actually a gradual improvement with more data. But if you do want to appeal to Lecun as an authority, then maybe you'll accept that these (re)tweets that clearly indicate he finds the insights from the paper to be valid: https://nitter.1d4.us/ylecun/status/1736479356917063847 https://nitter.1d4.us/ylecun/status/1736479356917063847 https://nitter.1d4.us/rao2z/status/1736464000836309259 https://nitter.1d4.us/rao2z/status/1736464000836309259 (retweeted) As for Timnit, I think you have your timeline confused. Model cards are what put her on the map for most general NLP researchers, which predates her difficulties at Google. 2018: Model cards paper was put on arXiv https://arxiv.org/abs/1810.03993 https://arxiv.org/abs/1810.03993 2019: Major ML organizations start using model cards https://github.com/openai/gpt-2/blob/master/model_card.md https://github.com/openai/gpt-2/blob/master/model_card.md 2020: Model cards become fairly standard https://blog.research.google/2020/07/introducing-model-card-toolkit-for.html https://blog.research.google/2020/07/introducing-model-card-... Dec 2020: Timnit is let go from the ethics team at Google https://www.bbc.com/news/technology-55187611 https://www.bbc.com/news/technology-55187611 EDIT:formatting
crowbahr 3y ago
Latest research shows emergent behavior is illusory. It doesn't preclude future emergence but currently models show 0 emergent behavior. To me the most interesting aspect of LLMs is the way that they reveal cognitive 0-days in humans. The human race needs patches to cognitive firmware to deal with predictive text... Which is a fascinating revelation to me. Sure it's backed up by psych analysis for decades but it's interesting to watch it play out on such a large scale.
- gitfan86 3y ago
  When a human makes a mistake it is a "cognitive 0-day" but when an LLM does something correctly it is "illusory"?
  - crowbahr 3y ago
    The cognitive 0-day is not the way that humans act like LLMs, it's the way humans anthropomorphize LLMs. It's the blind faith that LLMs do more than they do. The illusion of emergence is fact not fiction. The cognitive biases exposed by stochastic parrots are fact not fiction.
    - gitfan86 3y ago
      That is no different than saying beauty is only real if it is 100% natural. A woman who wears makeup and colors her hair is just an illusion of beauty. It is a philosophical argument to say that a machine isn't truly intelligent because it isn't using the same type of neural network as a human
      - discreteevent 3y ago
        Parent is saying that with something as sophisticated as intelligence it's not enough to say that if it behaves like a duck it's a duck (which is what your seem to be saying and which the parent calls a 0-day). There are some really good bulshitters who have led smart people into deep trouble. These bulshitters behaved really like ducks but they weren't ducks. The duck test just isn't good enough. The -1 day is where people say that because LLMs behave like humans then humans must be based on the same tech. I just wonder if these people have ever debugged a complex system only to discover that their initial model of how it worked was way off.
HarHarVeryFunny 3y ago
> I also don't understand the claims that it doesn't generalize. I currently use it to solve problems that I can absolutely guarantee were not in its training set, and it generalizes well enough. I also think that one of the easiest ways to get it to generalize better would simply be through giving it synthetic data which demonstrates the process of generalizing. I don't think what LLMs are currently doing is really generalizing, but rather: 1) Multiple occurrences of something in the dataset are mutually statistically reinforcing. This isn't generalization (abstraction) but rather reinforcement through repetition. 2) Multiple different statistical patterns are being recalled/combined in novel ways such that it seems able to "correctly" respond to things out of dataset, but really this only due to these novel combinations, not due to it having abstracted it's knowledge and applying a more general (or analogical) rule than present in it's individual training points.

jahnu 3y ago

> problems that I can absolutely guarantee were not in its training set Can you share the strongest example?

Jabrov 3y ago
Pretty much any coding problem in a unique or private codebase
- Der_Einzige 3y ago
  There is a difference between interpolation, which the majority of humans are performing daily with coding in private codebases, and genuine extrapolation, which is difficult to prove and difficult to find in high dimensional spaces. LLMs may not be able to easily extrapolate (and when it does it's due to high temperature), but they can interpolate extremely well, and most human growth and innovation today comes from novel interpolations, which are what LLMs are excellent at.
- jahnu 3y ago
  I asked for the strongest example the OP can share in order to evaluate their claim. If it's so obvious to the OP that generalisation is happening then it should be easy to provide a strong example, right?
- Xelynega 3y ago
  When I talk to people that use copilot to improve their coding workflow, what I often hear is that copilot can replace boilerplate but not any business logic specific to the problem being solved. It sounds like you have a different experience(copilot is useful for business logic). Do you have maybe any examples of what you mean by "any coding problems"? Is it similar to what I've heard previously(copilot can replace boilerplate) or have LLMs actually solved business problems for you in code?

nsagent 3y ago

I mentioned this to another commenter as well: You might want to reconsider your stance on emergent abilities in LLMs considering the NeurIPS 2023 best paper winner is titled: "Are Emergent Abilities of Large Language Models a Mirage?" https://arxiv.org/abs/2304.15004 https://arxiv.org/abs/2304.15004 https://blog.neurips.cc/2023/12/11/announcing-the-neurips-2023-paper-awards/ https://blog.neurips.cc/2023/12/11/announcing-the-neurips-20...

Der_Einzige 3y ago
Papers which get accepted with honors are not necessarily more truthful than papers which have been rejected. Yann LeCunn goes on twitter like any other grad student around NeurIPS or ICML/ICMR and bitterly complains when one of his (many) papers is rejected. Whose more likely to be correct here? Yann LeCunn (the TOP nlp scholar in our field by citations, who does claim that most emergent capabilities are real in other papers), or a NeurIPS best paper winner? My bet is on Yann. Also, consider that some work gets a lot of positivity not for the work itself, but for the people who wrote it. Timnit Gebaru's work was effectively ignored until she got famous for her spat with jeff dean at google. Her citations have exploded as a result, and I don't think that most in the field think that the "stochastic parrot" paper was especially good, and certainly not her other papers which include significant amounts of work dedicated to claiming that LLM training is really bad for the environment (despite a single jet taking AI researchers to conferences being worse for the environment than LLM training circa that paper being written was taking). Doesn't matter that the paper was wrong, it's now highly cited because you get brownie points for citing her work in grievance studies influenced subfields of AI.
- peteradio 3y ago
  Yann LeCun through Meta is incentivized towards maximizing capital return based on local maxima. That is how all business works, there is not really a direct incentive to pushing boundaries beyond what can be immediately monetized.
- nsagent 3y ago
  Please at least read the paper before appealing to authority. It is a well designed set of experiments that clearly demonstrates that the notion of a "phase change" (rapid shift in capabilities) as a popularized by many people claiming emergence is actually a gradual improvement with more data. But if you do want to appeal to Lecun as an authority, then maybe you'll accept that these (re)tweets that clearly indicate he finds the insights from the paper to be valid: https://nitter.1d4.us/ylecun/status/1736479356917063847 https://nitter.1d4.us/ylecun/status/1736479356917063847 https://nitter.1d4.us/rao2z/status/1736464000836309259 https://nitter.1d4.us/rao2z/status/1736464000836309259 (retweeted) As for Timnit, I think you have your timeline confused. Model cards are what put her on the map for most general NLP researchers, which predates her difficulties at Google. 2018: Model cards paper was put on arXiv https://arxiv.org/abs/1810.03993 https://arxiv.org/abs/1810.03993 2019: Major ML organizations start using model cards https://github.com/openai/gpt-2/blob/master/model_card.md https://github.com/openai/gpt-2/blob/master/model_card.md 2020: Model cards become fairly standard https://blog.research.google/2020/07/introducing-model-card-toolkit-for.html https://blog.research.google/2020/07/introducing-model-card-... Dec 2020: Timnit is let go from the ethics team at Google https://www.bbc.com/news/technology-55187611 https://www.bbc.com/news/technology-55187611 EDIT:formatting

crowbahr 3y ago

Latest research shows emergent behavior is illusory. It doesn't preclude future emergence but currently models show 0 emergent behavior. To me the most interesting aspect of LLMs is the way that they reveal cognitive 0-days in humans. The human race needs patches to cognitive firmware to deal with predictive text... Which is a fascinating revelation to me. Sure it's backed up by psych analysis for decades but it's interesting to watch it play out on such a large scale.

gitfan86 3y ago
When a human makes a mistake it is a "cognitive 0-day" but when an LLM does something correctly it is "illusory"?
- crowbahr 3y ago
  The cognitive 0-day is not the way that humans act like LLMs, it's the way humans anthropomorphize LLMs. It's the blind faith that LLMs do more than they do. The illusion of emergence is fact not fiction. The cognitive biases exposed by stochastic parrots are fact not fiction.
  - gitfan86 3y ago
    That is no different than saying beauty is only real if it is 100% natural. A woman who wears makeup and colors her hair is just an illusion of beauty. It is a philosophical argument to say that a machine isn't truly intelligent because it isn't using the same type of neural network as a human
    - discreteevent 3y ago
      Parent is saying that with something as sophisticated as intelligence it's not enough to say that if it behaves like a duck it's a duck (which is what your seem to be saying and which the parent calls a 0-day). There are some really good bulshitters who have led smart people into deep trouble. These bulshitters behaved really like ducks but they weren't ducks. The duck test just isn't good enough. The -1 day is where people say that because LLMs behave like humans then humans must be based on the same tech. I just wonder if these people have ever debugged a complex system only to discover that their initial model of how it worked was way off.

HarHarVeryFunny 3y ago

> I also don't understand the claims that it doesn't generalize. I currently use it to solve problems that I can absolutely guarantee were not in its training set, and it generalizes well enough. I also think that one of the easiest ways to get it to generalize better would simply be through giving it synthetic data which demonstrates the process of generalizing. I don't think what LLMs are currently doing is really generalizing, but rather: 1) Multiple occurrences of something in the dataset are mutually statistically reinforcing. This isn't generalization (abstraction) but rather reinforcement through repetition. 2) Multiple different statistical patterns are being recalled/combined in novel ways such that it seems able to "correctly" respond to things out of dataset, but really this only due to these novel combinations, not due to it having abstracted it's knowledge and applying a more general (or analogical) rule than present in it's individual training points.