We know regions associated with consciousness in the brain. A recent paper took that knowledge and did something kind of clever - they looked in species where t

We know regions associated with consciousness in the brain. A recent paper took that knowledge and did something kind of clever - they looked in species where that region was under development. Out of it came a lot of conjecture related to the region being used in the development of complex self-simulation motor control. It struck me that the intelligence theories of reinforcement learning have been encountering this same paradigm of self-simulation being necessary. A Tesla for example when it has to go down a street that also has another car needs to do a rollout not just for itself but the other car in order to actuate such that it can move in this motor control task. Not being able to model both agents makes it really trick to do planning. These tree search married to modeling problems seems to show up all over the place in the top performing architectures. For this reason I'm really suspect of the language model view of consciousness; it doesn't seem plausible. Or at least, I'm a little skeptical. In theory for a significantly large enough language model I don't see why you couldn't use a "loop unrolling" argument to posit that for some subset of the network there is an analogous computation to what my best guess about what consciousness is might look like. Worse for me is that when I strong man the argument I run into things like "the training task involves predicting the output of conscious entities and so a truly generalizing agent would be directed to learn just that unrolled loop". I'm not saying I think the language models are conscious, but I do see how reasonable people can arrive at those views. We probably need to write down a very hard definition of consciousness even if its wrong - or use terms that are better defined - because the ambiguity in the realm of things we don't fully understand makes this an annoying "words" problem rather than a concrete "facts" problem.