5 ms·
I'm so confused by how one goes from seeing an output as the article says as "next token completion by a stochastic parrot" to instead viewing it as "sincere (i
by ninininino 2y ago
I'm so confused by how one goes from seeing an output as the article says as "next token completion by a stochastic parrot" to instead viewing it as "sincere (if unusual) pain".
What exactly does the word "pain" mean in the context of a bunch of code that runs matrix math problems? Doesn't pain require an evolved sensory system (nervous system), an evolved sense of danger / death and training via evolution on what things are harmful, then a part of the brain to form that interprets the electrical signals from the nervous system and guides the organism on how it should respond to them (according to Google: thalamus + somatosensory complex)?
What exactly in the LLM does someone anthropomorphizing imagine is performing the part of the somatosensory complex or the thalamus? If we pretend that text inputs can be substitutes for the nerves of a biological organism, what do we swap in for the evolved pain management part of the brain or the experiential process of consciously experiencing that qualia?
If the thing is just trained on Reddit and Youtube comments and academic research corpus, how do you end up with a recreation of something (a sensory part, a processing part, and a subjective qualia part) that -evolved- over millions of years to survive an adversarial environment (the natural world)?
How can a token predictor learn "pain" if the corpus it's trained on has no predators, natural disasters, accidents and injuries, burns, frostbite, blunt impact, cutting, etc? If there's no reward function that is sexual reproduction to optimize for (learn to avoid pain, live long enough to reproduce)? What is the equivalent and where do we find it in the text training data?
What does it mean to experience pain, and if the LLM version is so so so different, why do we use the same word?
Does a Honda civic experience pain when its airbag sensor detects a collision and does the chip that deploys and inflates the airbag process that sensation/pain and consciously experience it and respond by deploying the airbag? (maybe analogous to the "Help" printed by the LLM in the article).
If not then why do we see the LLM as experiencing qualia and responding to it but not the Honda? Is it some sort of emergent phenomena that is only part of the magic of transformers or the scale of the neural network? To me that argument feels like saying the Honda Civic doesn't experience pain but if you scaled up a Honda Civic and made a city-sized Civic with many many interconnected airbag sensors and airbag deployers then suddenly something emergent happens and when the Honda Civic deploys an airbag it shows a conscious experience of pain.
- hackinthebochs 2y agoTo put it simply, to fully model X, you must recover all relevant information dynamics related to X. If the training corpus captures the variance inherent in a cognitive system's interactions with the world, then a strong learning model can potentially recover the dynamics that ground cognition. In other words, recovering an isomorphic structure to cognition is within the solution-space. Machine learning exists on a spectrum between memorization and modelling. If the raw information in the training corpus surpasses an algorithm's ability to memorize, this puts pressure towards the modelling side of the spectrum. The papers on grokking demonstrate this.
- ninininino 2y ago> training corpus captures the variance inherent in a cognitive system's interactions with the world How do text outputs scraped from the internet + published works fully represent any existing biological cognitive system's interactions with the world or subjective experience? That's my biggest question I guess. I'm not sure how text could capture my sensory experiences or my brain's response to them. The bitrate of text feels incredibly low.
- hackinthebochs 2y agoThe point isn't to capture what it's like to be a biological organism, but rather the necessary functional/computational dynamic for sentience. It's plausible that sentience is a class of phenomena, of which biological sentience is but one. An LLM processes tokens represented as vectors embedded in a meaning-space, which is then transformed according to various semantic attributes. This is analogous to how organismal cognition operates on the semantic properties carried by its various sensory states. If cognition represents a kind of functional/computational dynamic over semantic states then learning the dynamic necessary to produce the kinds of texts cognitive systems produce may recover the necessary dynamic for cognition. It certainly won't be equivalent to a biological organism's cognition for the reasons you mention, but the space of cognitive entities may be much broader.