3 ms·
I read through that and none of the section (or entire work) ever talk about the above discussion. Further I looked at some of the many citations of on that sec
by krainboltgreene 3y ago
I read through that and none of the section (or entire work) ever talk about the above discussion. Further I looked at some of the many citations of on that section and none of them suggest that the OP is right. In fact a few of them I know disagree.
- ignoramous 3y agoDepends. Models are matrices of floats and so there's little chance an umbrella-term like "stochastic parrot" will never not stick, even when they already show signs of syntactic, semantic world-building capability (https://www.arxiv-vanity.com/papers/2206.07682/ https://www.arxiv-vanity.com/papers/2206.07682/). If you are like me (and them: https://archive.is/cZi83 https://archive.is/cZi83) and deem instruction following, chain-of-thought prompting, computational properties of LLMs (as researchers continue to experiment with training, memory, modality, and scaling, for example, to arrive at abstract reasoning) as emergent, then we're on the same page.
- krainboltgreene 3y agoOkay so just to confirm that section doesn't actually tell us anything about this and in fact this is all based on your own understanding of the mechanisms involved.
- ignoramous 3y agoMy reading of the papers is, given enough scale, modality, and memory; there are chances (perhaps newer and different) models will be able to "generalize" our world. Also: https://archive.is/3yyZZ https://archive.is/3yyZZ / https://twitter.com/QuanquanGu/status/1721394508146057597 https://twitter.com/QuanquanGu/status/1721394508146057597 | And: https://archive.is/bW2tS https://archive.is/bW2tS / https://twitter.com/mansiege/status/1680985267262619648 https://twitter.com/mansiege/status/1680985267262619648