3 ms·

> Neural nets do seem to have some sort of emergent structural concept graph, in the case of LLMs it's largely informed by human language (because that's what t

by wahern 3y ago

> Neural nets do seem to have some sort of emergent structural concept graph, in the case of LLMs it's largely informed by human language (because that's what they're trained on.) To an extent, we can observe this empirically through their output even if the first principles are opaque. Alternatively, what you're seeing are the structures inherent within human culture as manifested through its literature[1], with LLMs simply being a new and useful tool which makes these structures more apparent. [1] And also its engineers' training choices

hprotagonist 3y ago
i buy this. cf the “lossy jpeg of the web” analogy; sure there’s structure: we put it there!
- pixl97 3y ago
  Then why wouldn't any LLM trained on real data have structure? I mean we don't live in the chaos realm, atoms make molecules, those molecules make structures, those structures make cells, organs, lifeforms.
  - hprotagonist 3y ago
    the degrees to which the (non)deterministic and (non)atomistic models of the universe have predictive power aside — yes, sure: you should expect a well trained NN to have an internal structure that approximates that of the statistical distribution of encoded features of the training data into the latent space of the model. that’s very nearly the definition of “trained”.
- enva2712 3y ago
  > lossy jpeg of web Knowledge is compression https://en.wikipedia.org/wiki/Hutter_Prize https://en.wikipedia.org/wiki/Hutter_Prize
  - hprotagonist 3y ago
    “if you can compress it this much it’s gotta be agi!” “no, we just made gzip better. again.”
    - enva2712 3y ago
      My comment wasn’t about agi, but out of curiosity what definition are you using for it? To be fair, I don’t see a categorical difference between human intelligence and compression either
TeMPOraL 3y ago
Alternatively, both are the same thing. The structures embedded in human culture, which the literature captures, are themselves manifestations of structures of our understanding on the world.
- wahern 3y ago
  I meant to address the notion that LLMs manifest emergent structures, perhaps derived from but qualitatively distinct and independent from the indisputably human intelligence-centered structures. Whether and to what extent human culture itself has emergent properties, including some sort of emergent intelligence, is a different question; indeed, I think a more interesting question than that of LLMs, though the dispute over the behavior and nature of LLMs certainly suggests these questions generally, and begs the question of what we mean wrt emergence, intelligence, etc. (My understanding of emergence comes from reading the seminal book, Complexity: The Emerging Science at the Edge of Order and Chaos, and some related literature. As far as I understand emergence, it's not really a close question of whether LLMs exhibit emergence.)

by wahern 3y ago

hprotagonist 3y ago

i buy this. cf the “lossy jpeg of the web” analogy; sure there’s structure: we put it there!

pixl97 3y ago
Then why wouldn't any LLM trained on real data have structure? I mean we don't live in the chaos realm, atoms make molecules, those molecules make structures, those structures make cells, organs, lifeforms.
- hprotagonist 3y ago
  the degrees to which the (non)deterministic and (non)atomistic models of the universe have predictive power aside — yes, sure: you should expect a well trained NN to have an internal structure that approximates that of the statistical distribution of encoded features of the training data into the latent space of the model. that’s very nearly the definition of “trained”.
enva2712 3y ago
> lossy jpeg of web Knowledge is compression https://en.wikipedia.org/wiki/Hutter_Prize https://en.wikipedia.org/wiki/Hutter_Prize
- hprotagonist 3y ago
  “if you can compress it this much it’s gotta be agi!” “no, we just made gzip better. again.”
  - enva2712 3y ago
    My comment wasn’t about agi, but out of curiosity what definition are you using for it? To be fair, I don’t see a categorical difference between human intelligence and compression either

TeMPOraL 3y ago

Alternatively, both are the same thing. The structures embedded in human culture, which the literature captures, are themselves manifestations of structures of our understanding on the world.

wahern 3y ago
I meant to address the notion that LLMs manifest emergent structures, perhaps derived from but qualitatively distinct and independent from the indisputably human intelligence-centered structures. Whether and to what extent human culture itself has emergent properties, including some sort of emergent intelligence, is a different question; indeed, I think a more interesting question than that of LLMs, though the dispute over the behavior and nature of LLMs certainly suggests these questions generally, and begs the question of what we mean wrt emergence, intelligence, etc. (My understanding of emergence comes from reading the seminal book, Complexity: The Emerging Science at the Edge of Order and Chaos, and some related literature. As far as I understand emergence, it's not really a close question of whether LLMs exhibit emergence.)