4 ms·

I'm curious in your statement, can you point to some papers where they addressed it?

ignoramous 3y ago
(not op) The section A Path Forward in Managing AI Risks by Bengio et al cites a few papers: https://managing-ai-risks.com/ https://managing-ai-risks.com/
- krainboltgreene 3y ago
  I read through that and none of the section (or entire work) ever talk about the above discussion. Further I looked at some of the many citations of on that section and none of them suggest that the OP is right. In fact a few of them I know disagree.
  - ignoramous 3y ago
    Depends. Models are matrices of floats and so there's little chance an umbrella-term like "stochastic parrot" will never not stick, even when they already show signs of syntactic, semantic world-building capability (https://www.arxiv-vanity.com/papers/2206.07682/ https://www.arxiv-vanity.com/papers/2206.07682/). If you are like me (and them: https://archive.is/cZi83 https://archive.is/cZi83) and deem instruction following, chain-of-thought prompting, computational properties of LLMs (as researchers continue to experiment with training, memory, modality, and scaling, for example, to arrive at abstract reasoning) as emergent, then we're on the same page.
    - krainboltgreene 3y ago
      Okay so just to confirm that section doesn't actually tell us anything about this and in fact this is all based on your own understanding of the mechanisms involved.
      - ignoramous 3y ago
        My reading of the papers is, given enough scale, modality, and memory; there are chances (perhaps newer and different) models will be able to "generalize" our world. Also: https://archive.is/3yyZZ https://archive.is/3yyZZ / https://twitter.com/QuanquanGu/status/1721394508146057597 https://twitter.com/QuanquanGu/status/1721394508146057597 | And: https://archive.is/bW2tS https://archive.is/bW2tS / https://twitter.com/mansiege/status/1680985267262619648 https://twitter.com/mansiege/status/1680985267262619648

(not op) The section A Path Forward in Managing AI Risks by Bengio et al cites a few papers: https://managing-ai-risks.com/ https://managing-ai-risks.com/

krainboltgreene 3y ago
I read through that and none of the section (or entire work) ever talk about the above discussion. Further I looked at some of the many citations of on that section and none of them suggest that the OP is right. In fact a few of them I know disagree.
- ignoramous 3y ago
  Depends. Models are matrices of floats and so there's little chance an umbrella-term like "stochastic parrot" will never not stick, even when they already show signs of syntactic, semantic world-building capability (https://www.arxiv-vanity.com/papers/2206.07682/ https://www.arxiv-vanity.com/papers/2206.07682/). If you are like me (and them: https://archive.is/cZi83 https://archive.is/cZi83) and deem instruction following, chain-of-thought prompting, computational properties of LLMs (as researchers continue to experiment with training, memory, modality, and scaling, for example, to arrive at abstract reasoning) as emergent, then we're on the same page.
  - krainboltgreene 3y ago
    Okay so just to confirm that section doesn't actually tell us anything about this and in fact this is all based on your own understanding of the mechanisms involved.
    - ignoramous 3y ago
      My reading of the papers is, given enough scale, modality, and memory; there are chances (perhaps newer and different) models will be able to "generalize" our world. Also: https://archive.is/3yyZZ https://archive.is/3yyZZ / https://twitter.com/QuanquanGu/status/1721394508146057597 https://twitter.com/QuanquanGu/status/1721394508146057597 | And: https://archive.is/bW2tS https://archive.is/bW2tS / https://twitter.com/mansiege/status/1680985267262619648 https://twitter.com/mansiege/status/1680985267262619648