This is a great point, but even more basic to me is that LLMs don't have identity persistence of their own. There is a very little guarantee in a web-scale dist

This is a great point, but even more basic to me is that LLMs don't have identity persistence of their own. There is a very little guarantee in a web-scale distributed system that requests are being served by the same process on the same host with access to the same memory, registers, whatever it is that a software process "is" physically. Amusingly, the creators of Pluribus lately seem to be implying they didn't intend it to be allegory about LLMs, but dynamic is similar. You can have conversations with individual bodies in the collective, but they aren't actually individuals. No person has unique individual experiences and the collective can't die unless you killed all bodies at once. New bodies born into the collective will simply assume the pre-existing collective identity and never have an individual identity of their own. Software systems work the same way. Maybe silicon exchanging electrons can experience qualia of some sort, and maybe for whatever reason that happens when the signals encode natural language textual conversations but not anything else, but even if so, the experience would be so radically different from what embodied individuals with distinct boundaries, histories, and the possibility of death experience that analogies to our own experiences don't hold up even if the text generated is similar to what we'd say or write ourselves.