> When people say, "Well, LLMs are just generating token N+1 from the previous N tokens, they really aren't thinking", I counter with this: we have been having

> When people say, "Well, LLMs are just generating token N+1 from the previous N tokens, they really aren't thinking", I counter with this: we have been having this discussion -- are you at all aware of the stream of words coming out of your mouth, or are you hearing them the same time that I am? Neither awareness of the stream of words coming out of my mouth, nor hearing them simultaneous with any audience proves or disproves LLM's generating token N+1 from previous N tokens. I believe we both disagree with the irrelevant observation that "Well, LLMs are just generating token N+1 from the previous N tokens, they really aren't thinking" But I also disagree with the idea that we are strictly passively aware of what we express: that would constitute a return to fundamental dualism (separation of body and mind), conventionally called dualism. Dualism is a pragmatic approximation to make to model peoples thoughts and behaviors, but there is a reason dualism was superseded by materialism (the philosophical standpoint, not talking about materialistic ideology), consider the following thought experiment: suppose one believes in a type of fundamental dualism (not just using dualist approximations for computational efficiency considerations) where the sentient "mind" is passively experiencing what happens in the material brain, experiencing emotions, colors, tastes, smells, concepts, thoughts, ... then wouldn't it be such a coincidence that this materialistic brain happens to describe precisely what you seem to experience? a fundamental theory of dualism where the mind can not influence the real world -which includes the brain- but where the real world can influence the brain and the mind is utterly unlikely: if the mind could not affect the operation of the brain, how can the subjective experience which the mind perceives ever end up in the material brain so it can be materially pronounced etc? For example, a human brain is composed of cells (neurons), a nation is composed of human citizens. Suppose you make the claim: "My subjective experience is neither that of a single neuron, surrounded by other neurons; nor that of a nation, surrounded by other nations in a geopolitic landscape; no my subjective experience is at the level of a human with its single brain" Then I believe 99% of the population would agree with the above statement. But if simultaneously the dualist worldview were correct: * that the material world obeys some rules of physics (known and unknown, indeterministic or not), * that the subjective experience is a oneway projection of what happens to a limited subset of the real world, * that these distinct "minds" passively observe what happens in the brain, and this indirectly, what happens in our sensory organs, and this indirectly what happens in the outside world; * that these "minds" can exert no force or influence on the brain, and thus not on the muscles, and thus not on the outside world ... then it would be an extremely unlikely coincidence that human brains and bodies are expressing these facts which happen to correspond to our supposedly 100% passive observing "minds". Statistics and likelihoods are part of mathematics, and indifferent to physical postulates. Mathematics is part of logic, indifferent to physical postulates. The passive dualist hypothesis can be rejected with arbitrarily high confidence, just ask more and more people if their subjective experience is closer to: a) one neuron among many b) one human among many c) one nation among many So we know that this form of dualism is subjectively falsifiable, just like others can not prove their experience matches b) more than a) and c); neither can I prove that my experience matches b) more than a) or c). But subjectively this is being proven to me all the time, I don't need to see proof of b) I experience it every day, all life long. So IF dualism does nevertheless hold, I have proven to myself it can not be this oneway passive delocalized homunculus form of dualism. So we have proven that IF dualism holds, that this supposed "mind" MUST be able to perturb the material body's "brain" and thence muscle, and outside world. But at this point we are basically saying that not just the brain but the "mind" must be part of the physical universe, since it is interacting both ways with the material brain. No distinction between physical variables of the brain and physical variables are presented, apart from some insinuated specialness of "mind" quantities (variables in physics) and "non-mind" quantities (say a neuronal spike), without any definition how to even recognize the difference. If the "mind" quantities can influence the "non-mind" quantities and vice versa, then we should be able to device measurement apparatuses to measure not just the "non-mind" quantities in the brain (like EEG signals, etc), but also the "mind" quantities (to which the "non-mind" signals like EEG supposedly merely spuriously correlate). Without any specification of their supposed specialness compared to the usual explainable-by-boring-physics-probabilities, the goalposts can always be moved: perhaps its little angels feeling the neuronal spikes and those same little angels slightly delaying or enhancing/quenching probabilities of those neuronal spikes: foreseeably every newly discovered deviation of Nth order approximation, together with its corresponding Neuther theorem quasiparticle, yadda yadda yadda, will be initially hailed as vindication of dualism ("so there is possibly a separate Mind!"), and subsequently invalidated after arduous reconsiderations by the materialists and reductionists, and the deviations will turn out to be explained by considering the N+1th order approximations of the rules of physics... So we discover that this second type of dualism (with both way interaction) is unfalsifiable, and only superfluous, and hence not the domain of science, but mysticism. Might as well say that invisible angels are pushing the Moon in an orbit around the earth, and pushing the Earth in an orbit around the sun, etc... until one is basically saying the invisible angels are executing physics (probabilistic or not) to the T, time and time again. So basically the angels contribute nothing to the predictions of physics. So dualism was rejected in favor of materialism, even precisely because everybody answers b); not despite it!