6 ms·
The article misses three critical points: 1. Conflates consciousness with "thinking" - LLMs may process information effectively without being conscious, but th
by adamzwasserman 1y ago
The article misses three critical points:
1. Conflates consciousness with "thinking" - LLMs may process information effectively without being conscious, but the article treats these as the same phenomenon
2. Ignores the cerebellum cases - We have documented cases of humans leading normal lives with little to no brain beyond a cerebellum, which contradicts simplistic "brain = deep learning" equivalences
3. Most damning: When you apply these exact same techniques to anything OTHER than language, the results are mediocre. Video generation still can't figure out basic physics (glass bouncing instead of shattering, ropes defying physics). Computer vision has been worked on since the 1960s - far longer than LLMs - yet it's nowhere near achieving what looks like "understanding."
The timeline is the smoking gun: vision had decades of head start, yet LLMs leapfrogged it in just a few years. That strongly suggests the "magic" is in language itself (which has been proven to be fractal and already heavily compressed/structured by human cognition) - NOT in the neural architecture.
We're not teaching machines to think.
We're teaching them to navigate a pre-existing map that was already built.
- eloisant 1y agoThis is why I'm very skeptical about the "Nobel prize level" claims. To win a Nobel prize you would have to produce something completely new. LLM will probably be able to reach a Ph.D. level of understanding existing research, but bringing something new is a different matter.
- adamzwasserman 1y agoLLMs do not understand anything. They have a very complex multidimensional "probability table" (more correctly a compressed geometric representation of token relationships) that they use to string together tokens (which have no semantic meaning), which then get converted to words that have semantic meaning to US, but not to the machine.
- KoolKat23 1y agoIn abstract we do the exact same thing
- tomfly 1y agoIt’s hard to believe this when the llm “knows” so much more then us yet still can not be creative outside its training distribution
- KoolKat23 1y agoWhen are we as humans creative outside our training data? It's very rare we actually discover something truly novel. This is often random, us stumbling onto it, brute force or purely by being at the right place at the right time. On the other hand, until it's proven it'd likely be considered a hallucination. You need to test something before you can dismiss it. (They did burn witches for discoveries back in the day, deemed witchcraft). We also reduce randomness and pre-train to avoid overfitting. Day to day human creative outputs as humans are actually less exciting when you think about it further, we build on pre-existing knowledge. No different to good prompt output with the right input. Humans are just more knowledgeable & smarter at the moment.
- adamzwasserman 1y agoThe LLM doesn't 'know' more than us - it has compressed more patterns from text than any human could process. That's not the same as knowledge. And yes, the training algorithms deliberately skew the distribution to maintain coherent output - without that bias toward seen patterns, it would generate nonsense. That's precisely why it can't be creative outside its training distribution: the architecture is designed to prevent novel combinations that deviate too far from learned patterns. Coherence and genuine creativity are in tension here
- adamzwasserman 1y agoPerhaps in practice as well. It is well-established that our interaction with language far exceeds what we are conscious of.
- tomfly 1y agoExactly. It’s been stated for a long time, before llms. For instance this paper https://home.csulb.edu/~cwallis/382/readings/482/searle.minds.brains.programs.bbs.1980.pdf https://home.csulb.edu/~cwallis/382/readings/482/searle.mind... Describes a translator who doesn’t know the language.
- DoctorOetker 1y agoConsider your human brain, and the full physical state, all the protons and neutrons some housed together in the same nucleus, some separate, together with all the electrons. Physics assigns probabilities to future states. Suppose you were in the middle of a conversation and about to express a next syllable (or token). That choice will depend on other choices ("what should I add next"), and further choices ("what is the best choice of words to express the thing I chose to express next etc. The probabilities are in principle calculable given a sufficiently detailed state. You are correct that LLM's correspond to a probability distribution (given you immediately corrected to say that this table is implicit and parametrized by a geometric token relationships.). But so does every expressor of language, humans included. The presence or absence of understanding can't be proven by mere association of with a "probability table", especially if such probability table is exactly expected from the perspective of physics, and if the models have continuously gained better and better performance by training them directly on human expressions!
- KoolKat23 1y agoGiven a random prompt, the overall probability of seeing a specific output string is almost zero, since there are astronomically many possible token sequences. The same goes for humans. Most awards are built on novel research built on pre-existing works. This a LLM is capable of doing.
- adamzwasserman 1y agoLLMs don't use 'overall probability' in any meaningful sense. During training, gradient descent creates highly concentrated 'gravity wells' of correlated token relationships - the probability distribution is extremely non-uniform, heavily weighted toward patterns seen in training data. The model isn't selecting from 'astronomically many possible sequences' with equal probability; it's navigating pre-carved channels in high-dimensional space. That's fundamentally different from novel discovery.
- KoolKat23 1y agoThat's exactly the same for humans in the real world. You're focusing too close, abstract up a level. Your point relates to the "micro" system functioning, not the wider "macro" result (think emergent capabilities).
- adamzwasserman 1y agoI'm afraid I'd need to see evidence before accepting that humans navigate 'pre-carved channels' in the same way LLMs do. Human learning involves direct interaction with physical reality, not just pattern matching on symbolic representations. Show me the equivalence or concede the point.
- KoolKat23 1y agoLanguage and math are a world model of physical reality. You could not read a book and make sense of it if this were not true. An apple falls to the ground because of? gravity. In real life this is the answer, I'm very sure the pre-carved channel will also lead to gravity.
- PaulDavisThe1st 1y ago> Conflates consciousness with "thinking" I don't see it. Got a quote that demonstrates this?
- thechao 1y agoI'm not really onboard with the whole LLM's-are-conscious thing. OTOH, I am totally onboard with the whole "homo sapiens exterminated every other intelligent hominid and maybe — just maybe — we're not very nice to other intelligences". So, I try not to let my inborn genetic predisposition to exterminate other intelligence pseudo-hominids color my opinions too much.
- adamzwasserman 1y agoIt's a dog eat dog world for sure. It does in fact seem that a part of intelligence is using it to compete ruthlessly with other intelligences.
- adamzwasserman 1y agoExactly. Notable by its absence.
- nearbuy 1y agoCan you explain #2? What does the part of the brain that's primarily for balance and motor control tell us about deep learning?
- adamzwasserman 1y agoMy mistake thx. I meant "despite having no, or close to no, brain beyond a cerebellum"
- nearbuy 1y agoAre there any cases like that? I've never heard of someone functioning normally with little or no brain beyond a cerebellum.
- adamzwasserman 1y agohttps://npr.org/sections/health-shots/2015/03/16/392789753/a-man-s-incomplete-brain-reveals-cerebellum-s-role-in-thought-and-emotion https://npr.org/sections/health-shots/2015/03/16/392789753/a... https://irishtimes.com/news/remarkable-story-of-maths-genius-who-had-almost-no-brain-1.1026845 https://irishtimes.com/news/remarkable-story-of-maths-genius... https://biology.stackexchange.com/questions/64017/what-sections-of-the-brain-can-be-removed-without-causing-instant-death https://biology.stackexchange.com/questions/64017/what-secti... https://cbc.ca/radio/asithappens/as-it-happens-thursday-edition-1.3679117/scientists-research-man-missing-90-of-his-brain-who-leads-a-normal-life-1.3679125 https://cbc.ca/radio/asithappens/as-it-happens-thursday-edit...
- nearbuy 1y agoThe first article is about someone missing a cerebellum, not part of their cerebrum. That's the motor and balance part of the brain, and as you might expect, the subject of the article has deficits in motor control and balance. The Biology StackExchange answer just says that frontal lobotomies don't kill you. It doesn't say that lobotomized people function normally. The other two articles are just misreporting on hydrocephalus. This is a condition where fluid build-up compresses the brain tissue, making it appear like a large part of the brain is missing in CT scans. The pressure from the fluid is actually compressing the brain. While it can damage the brain, there is no way to tell from the scans how much, if any, brain matter was destroyed. Hydrocephalus usually causes death or severe deficits, but occasionally it doesn't. Even assuming though that it were all true and people could function normally with little or no brain, that doesn't really tell us anything about LLMs, but rather just uppends all of neuroscience. It would seem to imply the brain isn't doing the thinking and perhaps we have something else like an intangible soul.
- kenjackson 1y ago"vision had decades of head start, yet LLMs leapfrogged it in just a few years." From an evolutionary perspective though vision had millions of years head start over written language. Additionally, almost all animals have quite good vision mechanisms, but very few do any written communication. Behaviors that map to intelligence don't emerge concurrently. It may well be there are different forms of signals/sensors/mechanical skills that contribute to emergence of different intelligences. It really feels more and more like we should recast AGI as Artificial Human Intelligence Likeness (AHIL).
- adamzwasserman 1y agoFrom a terminology point of view, I absolutely agree. Human-likeness is what most people mean when they talk about AGI. Calling it what it is would clarify a lot of the discussions around it. However I am clear that I do not believe that this will ever happen, and I see no evidence to convince that that there is even a possibility that it will. I think that Wittgenstein had it right when he said: "If a lion could speak, we could not understand him."
- andoando 1y ago>I think that Wittgenstein had it right when he said: "If a lion could speak, we could not understand him." Why would we not? We live in the same physical world and encounter the same problems.
- adamzwasserman 1y agoYou're actually proving Wittgenstein's point. We share the same physical world, but we don't encounter the same problems. A lion's concerns - territory, hunting, pride hierarchy - are fundamentally different from ours: mortgages, meaning, relationships. And here's the kicker: you don't even fully understand me, and I'm human. What makes you think you'd understand a lion?
- beeflet 1y ago
- bjourne 1y ago> 1. Conflates consciousness with "thinking" - LLMs may process information effectively without being conscious, but the article treats these as the same phenomenon There is NO WAY you can define "consciousness" in such a non-tautological, non-circular way that it includes all humans but excludes all LLMs.
- adamzwasserman 1y agoYou could have stopped here: "There is NO WAY you can define "consciousness"
- beeflet 1y agoWhy not? Consciousness is a state of self-awareness.
- bena 1y agoTo paraphrase Jean Luc Picard: Am I conscious? Why? Can you prove that I am conscious?
- beeflet 1y agoconsciousness is the feeling of self awareness. I suppose you could prove it as much as any other feeling, by observing the way that people behave
- inglor_cz 1y agoA robot can certainly be programmed to behave in a self-aware way, but making a conclusion about its actual self-awareness would be unfounded. In general, behaviorism wasn't a very productive theory in humans and animals either.
- beeflet 1y agoBy behaving in a self-aware way, it practices self awareness. It would only be unfounded if the robot is programmed in a way that seemingly appears to be self-aware but actually isn't (It would need to occasionally act in a non-self aware way, like a manchurian candidate). But if you keep increasing scrutiny, it converges on being self aware because the best way to appear self-aware is to be self-aware. It's not clear to me what the intrinsic goals of a robot would be if it did practice self-awareness in the first place. But in living things it's to grow and reproduce.
- penteract 1y agoThere's a whole paragraph in the article which says basically the same as your point 3 ( "glass bouncing, instead of shattering, and ropes defying physics" is literally a quote from the article). I don't see how you can claim the article missed it.
- adamzwasserman 1y agothe article misses the significance of it.
- pevansgreenwood 1y ago[dead]
- aucisson_masque 1y ago> 2. Ignores the cerebellum cases - We have documented cases of humans leading normal lives with little to no brain beyond a cerebellum, which contradicts simplistic "brain = deep learning" equivalences I went to look for it on Google but couldn't find much. Could you provide a link or something to learn more about ? I found numerous cases of people living without cerebellum but I fail to see how it would justify your reasoning.
- adamzwasserman 1y agohttps://npr.org/sections/health-shots/2015/03/16/392789753/a-man-s-incomplete-brain-reveals-cerebellum-s-role-in-thought-and-emotion https://npr.org/sections/health-shots/2015/03/16/392789753/a... https://irishtimes.com/news/remarkable-story-of-maths-genius-who-had-almost-no-brain-1.1026845 https://irishtimes.com/news/remarkable-story-of-maths-genius... https://biology.stackexchange.com/questions/64017/what-sections-of-the-brain-can-be-removed-without-causing-instant-death https://biology.stackexchange.com/questions/64017/what-secti... https://cbc.ca/radio/asithappens/as-it-happens-thursday-edition-1.3679117/scientists-research-man-missing-90-of-his-brain-who-leads-a-normal-life-1.3679125 https://cbc.ca/radio/asithappens/as-it-happens-thursday-edit...
- deleted 1y ago[deleted]
- bonsai_spool 1y agoYour first example is someone without a cerebellum which is not like the others. The other examples are people with compressed neural tissue but that is not the same as never having the tissue. A being with only a cerebellum could not behave like a human.
- adamzwasserman 1y agoYou're right - I mixed up cerebellum/cerebrum/cortex terminology. My bad. The cases I'm referencing are hydrocephalus patients with severely compressed cerebral tissue who maintained normal cognitive function. The point about structural variation not precluding consciousness stands.
- KoolKat23 1y ago1. Consciousness itself is probably just an illusion, a phenomena/name of something that occurs when you bunch thinking together. Think of this objectively and base it on what we know of the brain. It literally is working off of what hardware we have, there's no magic. 2. That's just a well adapted neural network (I suspect more brain is left than you let on). Multimodal model making the most of its limited compute and whatever gpio it has. 3. Humans navigate a pre-existing map that is already built. We can't understand things in other dimensions and need to abstract this. We're mediocre at computation. I know there's people that like to think humans should always be special.
- estearum 1y ago> Consciousness itself is probably just an illusion This is a major cop-out. The very concept of "illusion" implies a consciousness (a thing that can be illuded). I think you've maybe heard that sense of self is an illusion and you're mistakenly applying that to consciousness, which is quite literally the only thing in the universe we can be certain is not an illusion. The existence of one's own consciousness is the only thing they cannot possibly be illuded about (note: the contents of said consciousness are fully up for grabs)
- KoolKat23 1y agoI mean peoples perception of it being a thing rather than a set of systems. But if that's your barometer, I'll say models are conscious. They may not have proper agency yet. But they are conscious.
- adamzwasserman 1y ago1. 'Probably just an illusion' is doing heavy lifting here. Either provide evidence or admit this is speculation. You can't use an unproven claim about consciousness to dismiss concerns about conflating it with text generation. 2. Yes, there are documented cases of people with massive cranial cavities living normal lives. https://x.com/i/status/1728796851456156136 https://x.com/i/status/1728796851456156136. The point isn't that they have 'just enough' brain. it's that massive structural variation doesn't preclude function, which undermines simplistic 'right atomic arrangement = consciousness' claims. 3. You're equivocating. Humans navigate maps built by other humans through language. We also directly interact with physical reality and create new maps from that interaction. LLMs only have access to the maps - they can't taste coffee, stub their toe, or run an experiment. That's the difference.