I think at this point the conceptual error most people are making is thinking of GPT-X as a “mind”, which you can talk to and which has some sort of stable char

I think at this point the conceptual error most people are making is thinking of GPT-X as a “mind”, which you can talk to and which has some sort of stable character. It’s better modeled as a simulator, with the prompt as short term memory specifying what is being generated, and the simulator having the ability to simulate somewhat-intelligent characters. Interestingly I think giving a more persistent sense of self (including memories, and perhaps rumination) will be key to preventing an AI agent from being prompt-hijacked into simulating a new personality.