6 ms·
There's actually a somewhat reasonable analogy to human cognitive processes here, I think, in the sense that humans tend to form concepts defined by their conne
by lukev 3y ago
There's actually a somewhat reasonable analogy to human cognitive processes here, I think, in the sense that humans tend to form concepts defined by their connectivity to other concepts (c.f. Ferdinand de Saussure & structuralism).
Human brains are also a "black box" in the sense that you can't scan/dissect one to build a concept graph.
Neural nets do seem to have some sort of emergent structural concept graph, in the case of LLMs it's largely informed by human language (because that's what they're trained on.) To an extent, we can observe this empirically through their output even if the first principles are opaque.
- wahern 3y ago> Neural nets do seem to have some sort of emergent structural concept graph, in the case of LLMs it's largely informed by human language (because that's what they're trained on.) To an extent, we can observe this empirically through their output even if the first principles are opaque. Alternatively, what you're seeing are the structures inherent within human culture as manifested through its literature[1], with LLMs simply being a new and useful tool which makes these structures more apparent. [1] And also its engineers' training choices
- hprotagonist 3y agoi buy this. cf the “lossy jpeg of the web” analogy; sure there’s structure: we put it there!
- pixl97 3y agoThen why wouldn't any LLM trained on real data have structure? I mean we don't live in the chaos realm, atoms make molecules, those molecules make structures, those structures make cells, organs, lifeforms.
- hprotagonist 3y agothe degrees to which the (non)deterministic and (non)atomistic models of the universe have predictive power aside — yes, sure: you should expect a well trained NN to have an internal structure that approximates that of the statistical distribution of encoded features of the training data into the latent space of the model. that’s very nearly the definition of “trained”.
- enva2712 3y ago> lossy jpeg of web Knowledge is compression https://en.wikipedia.org/wiki/Hutter_Prize https://en.wikipedia.org/wiki/Hutter_Prize
- hprotagonist 3y ago“if you can compress it this much it’s gotta be agi!” “no, we just made gzip better. again.”
- enva2712 3y agoMy comment wasn’t about agi, but out of curiosity what definition are you using for it? To be fair, I don’t see a categorical difference between human intelligence and compression either
- TeMPOraL 3y agoAlternatively, both are the same thing. The structures embedded in human culture, which the literature captures, are themselves manifestations of structures of our understanding on the world.
- wahern 3y agoI meant to address the notion that LLMs manifest emergent structures, perhaps derived from but qualitatively distinct and independent from the indisputably human intelligence-centered structures. Whether and to what extent human culture itself has emergent properties, including some sort of emergent intelligence, is a different question; indeed, I think a more interesting question than that of LLMs, though the dispute over the behavior and nature of LLMs certainly suggests these questions generally, and begs the question of what we mean wrt emergence, intelligence, etc. (My understanding of emergence comes from reading the seminal book, Complexity: The Emerging Science at the Edge of Order and Chaos, and some related literature. As far as I understand emergence, it's not really a close question of whether LLMs exhibit emergence.)
- Kapura 3y agoThis gets at the fundamental issue I have with AI: We're trying to get machines to think, but we only have the barest understanding of how we, as humans, think. The concept of a "neuron" that can activate other neurons comes straight from real neurology, but given how complex the human brain is, it's no surprise that we can only create something that is fundamentally "lesser." However, I think that real neurology and machine-learning can be mutually reinforcing fields: structures discovered in the one could be applied to the other, and vice versa. But thinking we can create "AGI" without first increasing our understanding of "wet" neural nets is the height of hubris.
- sxg 3y agoAGI and human intelligence don't necessarily have to work the same way. While many of us assume that they must be very similar, I don't think there's any basis for that assumption.
- swatcoder 3y agoThat's correct that they don't need to work the same way. I would say most people that have earnestly thought about it assume they won't. But given the exponentially greater chemical, biological, and neurological complexity of human brains, the millions of years of evolutionary "pre-training" that they get imbued with, and the years of constant multimodal sensory input required for their culmination in what's called human intelligence, it takes an extremely bold assumption to believe that equivalent ends can be met with a stream of fmadds driving through in an array of transistors. It's hard to overstate how big a gulf there is in both the hardware and software between living systems and what we currently see in our silicon projects. Neural network learning in general and LLM's in particular are like crude paper airplanes compared to the flight capabilities of a dextrous bird. You can kinda squint and see that they both move some distance through the air, and even marvel at it for a bit, but there's a long long long long way to go.
- kelseyfrog 3y agoAnd yet no bird ever broke the sound barrier. Humans make things that exceed anything in nature. Thought will only be another example.