4 ms·

Even in the best of cases, our perceptions and interpretations of those perceptions are often dramatically flawed. Add on top of that neural atrophy, loss of ox

Even in the best of cases, our perceptions and interpretations of those perceptions are often dramatically flawed. Add on top of that neural atrophy, loss of oxygen, accumulation of senescent cells, poor clearance of waste products ... and what comes out is often going to be gibberish, as the remaining working parts of the mind attempts to construct a coherent narrative from the broken fragments of sensory input and failing memories. I got interested in consciousness 35 years ago or so when I read Oliver Sacks' "The Man Who Mistook His Wife For A Hat". Although the various people depicted in the stories had some physical deficit due to a trauma of some kind, it vividly demonstrated how we rationalize our way through the world more than we reason our way through it. Our conscious mind is much more of a facade than we typically imagine. When people say, "Well, LLMs are just generating token N+1 from the previous N tokens, they really aren't thinking", I counter with this: we have been having this discussion -- are you at all aware of the stream of words coming out of your mouth, or are you hearing them the same time that I am? Yes, sometimes we have deliberate thought where we rehearse different lines of reasoning before uttering something, but 98% of the time we are spewing just like LLMs do. And when we do engage in deliberate thought, each of those trial sentences again just appears without consideration; we are simply post-hoc picking the one that feels best.

Lammy 2y ago
> Add on top of that neural atrophy, loss of oxygen, accumulation of senescent cells, poor clearance of waste products I think the enjoyable thing about recreational drugs is the way they alter the normally-imperceptible boundaries between body/mind/Self. It provides an opportunity to understand what each of those things really are and what each contributes to my sense of being. I don't see these examples as all too different even if they are failure modes :)
- lostmsu 2y ago
  So what did you understand that you did not know before? Many people make the "opportunity to understand" claim, but I'm yet to hear anything meaningful coming out. Are you familiar with this joke: A guy decides to do drugs and keeps a notepad nearby to write down any profound revelations he has while high. The next morning he checks the notepad and finds he only wrote one thing: "The water is wet."
  - nextaccountic 2y ago
    Here's this famous question (I can't readily find a source so I will say as I remember) There is a scientist that studies color. She knows everything about it that can be possibly known. She knows the colors of everyday objects, she knows how colors work, how light's wavelength affects its transmission, etc etc. She is also fully colorblind (achromatopsia). One day, she finally can see colors somehow. She isn't colorblind anymore. The question is: now she can finally see the stuff she knew only intellectually, did she learn anything new?
    - lostmsu 2y ago
      Of course, she instantly sees color details of nearby objects down to a millimeter, so shit ton of new information. Bringing this back to the joke she can now look at a colored strip and write down the colors in the order they appear on it. But the insinuation that drugs have similar effect is easily disproved by continued inability of the junkie to write down any new information. Again, nobody dared to answer GGP question about examples of any revelations.
DoctorOetker 2y ago
> When people say, "Well, LLMs are just generating token N+1 from the previous N tokens, they really aren't thinking", I counter with this: we have been having this discussion -- are you at all aware of the stream of words coming out of your mouth, or are you hearing them the same time that I am? Neither awareness of the stream of words coming out of my mouth, nor hearing them simultaneous with any audience proves or disproves LLM's generating token N+1 from previous N tokens. I believe we both disagree with the irrelevant observation that "Well, LLMs are just generating token N+1 from the previous N tokens, they really aren't thinking" But I also disagree with the idea that we are strictly passively aware of what we express: that would constitute a return to fundamental dualism (separation of body and mind), conventionally called dualism. Dualism is a pragmatic approximation to make to model peoples thoughts and behaviors, but there is a reason dualism was superseded by materialism (the philosophical standpoint, not talking about materialistic ideology), consider the following thought experiment: suppose one believes in a type of fundamental dualism (not just using dualist approximations for computational efficiency considerations) where the sentient "mind" is passively experiencing what happens in the material brain, experiencing emotions, colors, tastes, smells, concepts, thoughts, ... then wouldn't it be such a coincidence that this materialistic brain happens to describe precisely what you seem to experience? a fundamental theory of dualism where the mind can not influence the real world -which includes the brain- but where the real world can influence the brain and the mind is utterly unlikely: if the mind could not affect the operation of the brain, how can the subjective experience which the mind perceives ever end up in the material brain so it can be materially pronounced etc? For example, a human brain is composed of cells (neurons), a nation is composed of human citizens. Suppose you make the claim: "My subjective experience is neither that of a single neuron, surrounded by other neurons; nor that of a nation, surrounded by other nations in a geopolitic landscape; no my subjective experience is at the level of a human with its single brain" Then I believe 99% of the population would agree with the above statement. But if simultaneously the dualist worldview were correct: * that the material world obeys some rules of physics (known and unknown, indeterministic or not), * that the subjective experience is a oneway projection of what happens to a limited subset of the real world, * that these distinct "minds" passively observe what happens in the brain, and this indirectly, what happens in our sensory organs, and this indirectly what happens in the outside world; * that these "minds" can exert no force or influence on the brain, and thus not on the muscles, and thus not on the outside world ... then it would be an extremely unlikely coincidence that human brains and bodies are expressing these facts which happen to correspond to our supposedly 100% passive observing "minds". Statistics and likelihoods are part of mathematics, and indifferent to physical postulates. Mathematics is part of logic, indifferent to physical postulates. The passive dualist hypothesis can be rejected with arbitrarily high confidence, just ask more and more people if their subjective experience is closer to: a) one neuron among many b) one human among many c) one nation among many So we know that this form of dualism is subjectively falsifiable, just like others can not prove their experience matches b) more than a) and c); neither can I prove that my experience matches b) more than a) or c). But subjectively this is being proven to me all the time, I don't need to see proof of b) I experience it every day, all life long. So IF dualism does nevertheless hold, I have proven to myself it can not be this oneway passive delocalized homunculus form of dualism. So we have proven that IF dualism holds, that this supposed "mind" MUST be able to perturb the material body's "brain" and thence muscle, and outside world. But at this point we are basically saying that not just the brain but the "mind" must be part of the physical universe, since it is interacting both ways with the material brain. No distinction between physical variables of the brain and physical variables are presented, apart from some insinuated specialness of "mind" quantities (variables in physics) and "non-mind" quantities (say a neuronal spike), without any definition how to even recognize the difference. If the "mind" quantities can influence the "non-mind" quantities and vice versa, then we should be able to device measurement apparatuses to measure not just the "non-mind" quantities in the brain (like EEG signals, etc), but also the "mind" quantities (to which the "non-mind" signals like EEG supposedly merely spuriously correlate). Without any specification of their supposed specialness compared to the usual explainable-by-boring-physics-probabilities, the goalposts can always be moved: perhaps its little angels feeling the neuronal spikes and those same little angels slightly delaying or enhancing/quenching probabilities of those neuronal spikes: foreseeably every newly discovered deviation of Nth order approximation, together with its corresponding Neuther theorem quasiparticle, yadda yadda yadda, will be initially hailed as vindication of dualism ("so there is possibly a separate Mind!"), and subsequently invalidated after arduous reconsiderations by the materialists and reductionists, and the deviations will turn out to be explained by considering the N+1th order approximations of the rules of physics... So we discover that this second type of dualism (with both way interaction) is unfalsifiable, and only superfluous, and hence not the domain of science, but mysticism. Might as well say that invisible angels are pushing the Moon in an orbit around the earth, and pushing the Earth in an orbit around the sun, etc... until one is basically saying the invisible angels are executing physics (probabilistic or not) to the T, time and time again. So basically the angels contribute nothing to the predictions of physics. So dualism was rejected in favor of materialism, even precisely because everybody answers b); not despite it!
BLKNSLVR 2y ago
> Oliver Sacks' "The Man Who Mistook His Wife For A Hat" Amazing book. Made me entirely reconsider how I think about human behaviour and "how they got like they did". Should be required reading for being a human living in a society.
johngossman 2y ago
Sacks also wrote a book called "Hallucinations," which is excellent and highly relevant to this article. If any of you have loved ones who are starting to hallucinate or experience visual artifacts of the type associated with Charles Benoit Syndrome, I highly recommend reading this.
deleted 2y ago
[deleted]
TriNetra 2y ago
It's interesting to reflect upon who is "I" presented with these experiences. Brain might be hallucinating in a given moment due to infection or something. But the witness "I" is ever-intact experiencing come what may.
nis0s 2y ago
> Our conscious mind is much more of a facade than we typically imagine. It patently is not, but I can see how it’s easy to believe it to be so. When you juxtapose the behavior of self-conscious organisms against reactive behavior, such as that observed during sentience-level consciousness, it’s easier to tease apart properties inherent to the conscious mind. For example, an LLM is going to have a hard time adjusting its behavior in an anticipatory manner to another organism, as their capacity for self-learning is currently limited. But even relatively simple-minded animals with relatively higher-level cognition will adjust responses not only based on prompts, but also on their own nascent understanding of what’s expected behavior given a situation by continually adapting and forecasting, which is different from simply forecasting. I got interested in LLMs because of all the hype around their capabilities, but even simpler-than-human minds are doing much more complex stuff than generating the next probable step.

Lammy 2y ago

> Add on top of that neural atrophy, loss of oxygen, accumulation of senescent cells, poor clearance of waste products I think the enjoyable thing about recreational drugs is the way they alter the normally-imperceptible boundaries between body/mind/Self. It provides an opportunity to understand what each of those things really are and what each contributes to my sense of being. I don't see these examples as all too different even if they are failure modes :)

lostmsu 2y ago
So what did you understand that you did not know before? Many people make the "opportunity to understand" claim, but I'm yet to hear anything meaningful coming out. Are you familiar with this joke: A guy decides to do drugs and keeps a notepad nearby to write down any profound revelations he has while high. The next morning he checks the notepad and finds he only wrote one thing: "The water is wet."
- nextaccountic 2y ago
  Here's this famous question (I can't readily find a source so I will say as I remember) There is a scientist that studies color. She knows everything about it that can be possibly known. She knows the colors of everyday objects, she knows how colors work, how light's wavelength affects its transmission, etc etc. She is also fully colorblind (achromatopsia). One day, she finally can see colors somehow. She isn't colorblind anymore. The question is: now she can finally see the stuff she knew only intellectually, did she learn anything new?
  - lostmsu 2y ago
    Of course, she instantly sees color details of nearby objects down to a millimeter, so shit ton of new information. Bringing this back to the joke she can now look at a colored strip and write down the colors in the order they appear on it. But the insinuation that drugs have similar effect is easily disproved by continued inability of the junkie to write down any new information. Again, nobody dared to answer GGP question about examples of any revelations.

DoctorOetker 2y ago

> When people say, "Well, LLMs are just generating token N+1 from the previous N tokens, they really aren't thinking", I counter with this: we have been having this discussion -- are you at all aware of the stream of words coming out of your mouth, or are you hearing them the same time that I am? Neither awareness of the stream of words coming out of my mouth, nor hearing them simultaneous with any audience proves or disproves LLM's generating token N+1 from previous N tokens. I believe we both disagree with the irrelevant observation that "Well, LLMs are just generating token N+1 from the previous N tokens, they really aren't thinking" But I also disagree with the idea that we are strictly passively aware of what we express: that would constitute a return to fundamental dualism (separation of body and mind), conventionally called dualism. Dualism is a pragmatic approximation to make to model peoples thoughts and behaviors, but there is a reason dualism was superseded by materialism (the philosophical standpoint, not talking about materialistic ideology), consider the following thought experiment: suppose one believes in a type of fundamental dualism (not just using dualist approximations for computational efficiency considerations) where the sentient "mind" is passively experiencing what happens in the material brain, experiencing emotions, colors, tastes, smells, concepts, thoughts, ... then wouldn't it be such a coincidence that this materialistic brain happens to describe precisely what you seem to experience? a fundamental theory of dualism where the mind can not influence the real world -which includes the brain- but where the real world can influence the brain and the mind is utterly unlikely: if the mind could not affect the operation of the brain, how can the subjective experience which the mind perceives ever end up in the material brain so it can be materially pronounced etc? For example, a human brain is composed of cells (neurons), a nation is composed of human citizens. Suppose you make the claim: "My subjective experience is neither that of a single neuron, surrounded by other neurons; nor that of a nation, surrounded by other nations in a geopolitic landscape; no my subjective experience is at the level of a human with its single brain" Then I believe 99% of the population would agree with the above statement. But if simultaneously the dualist worldview were correct: * that the material world obeys some rules of physics (known and unknown, indeterministic or not), * that the subjective experience is a oneway projection of what happens to a limited subset of the real world, * that these distinct "minds" passively observe what happens in the brain, and this indirectly, what happens in our sensory organs, and this indirectly what happens in the outside world; * that these "minds" can exert no force or influence on the brain, and thus not on the muscles, and thus not on the outside world ... then it would be an extremely unlikely coincidence that human brains and bodies are expressing these facts which happen to correspond to our supposedly 100% passive observing "minds". Statistics and likelihoods are part of mathematics, and indifferent to physical postulates. Mathematics is part of logic, indifferent to physical postulates. The passive dualist hypothesis can be rejected with arbitrarily high confidence, just ask more and more people if their subjective experience is closer to: a) one neuron among many b) one human among many c) one nation among many So we know that this form of dualism is subjectively falsifiable, just like others can not prove their experience matches b) more than a) and c); neither can I prove that my experience matches b) more than a) or c). But subjectively this is being proven to me all the time, I don't need to see proof of b) I experience it every day, all life long. So IF dualism does nevertheless hold, I have proven to myself it can not be this oneway passive delocalized homunculus form of dualism. So we have proven that IF dualism holds, that this supposed "mind" MUST be able to perturb the material body's "brain" and thence muscle, and outside world. But at this point we are basically saying that not just the brain but the "mind" must be part of the physical universe, since it is interacting both ways with the material brain. No distinction between physical variables of the brain and physical variables are presented, apart from some insinuated specialness of "mind" quantities (variables in physics) and "non-mind" quantities (say a neuronal spike), without any definition how to even recognize the difference. If the "mind" quantities can influence the "non-mind" quantities and vice versa, then we should be able to device measurement apparatuses to measure not just the "non-mind" quantities in the brain (like EEG signals, etc), but also the "mind" quantities (to which the "non-mind" signals like EEG supposedly merely spuriously correlate). Without any specification of their supposed specialness compared to the usual explainable-by-boring-physics-probabilities, the goalposts can always be moved: perhaps its little angels feeling the neuronal spikes and those same little angels slightly delaying or enhancing/quenching probabilities of those neuronal spikes: foreseeably every newly discovered deviation of Nth order approximation, together with its corresponding Neuther theorem quasiparticle, yadda yadda yadda, will be initially hailed as vindication of dualism ("so there is possibly a separate Mind!"), and subsequently invalidated after arduous reconsiderations by the materialists and reductionists, and the deviations will turn out to be explained by considering the N+1th order approximations of the rules of physics... So we discover that this second type of dualism (with both way interaction) is unfalsifiable, and only superfluous, and hence not the domain of science, but mysticism. Might as well say that invisible angels are pushing the Moon in an orbit around the earth, and pushing the Earth in an orbit around the sun, etc... until one is basically saying the invisible angels are executing physics (probabilistic or not) to the T, time and time again. So basically the angels contribute nothing to the predictions of physics. So dualism was rejected in favor of materialism, even precisely because everybody answers b); not despite it!

BLKNSLVR 2y ago

> Oliver Sacks' "The Man Who Mistook His Wife For A Hat" Amazing book. Made me entirely reconsider how I think about human behaviour and "how they got like they did". Should be required reading for being a human living in a society.

johngossman 2y ago

Sacks also wrote a book called "Hallucinations," which is excellent and highly relevant to this article. If any of you have loved ones who are starting to hallucinate or experience visual artifacts of the type associated with Charles Benoit Syndrome, I highly recommend reading this.

deleted 2y ago

[deleted]

TriNetra 2y ago

It's interesting to reflect upon who is "I" presented with these experiences. Brain might be hallucinating in a given moment due to infection or something. But the witness "I" is ever-intact experiencing come what may.

nis0s 2y ago

> Our conscious mind is much more of a facade than we typically imagine. It patently is not, but I can see how it’s easy to believe it to be so. When you juxtapose the behavior of self-conscious organisms against reactive behavior, such as that observed during sentience-level consciousness, it’s easier to tease apart properties inherent to the conscious mind. For example, an LLM is going to have a hard time adjusting its behavior in an anticipatory manner to another organism, as their capacity for self-learning is currently limited. But even relatively simple-minded animals with relatively higher-level cognition will adjust responses not only based on prompts, but also on their own nascent understanding of what’s expected behavior given a situation by continually adapting and forecasting, which is different from simply forecasting. I got interested in LLMs because of all the hype around their capabilities, but even simpler-than-human minds are doing much more complex stuff than generating the next probable step.