I'm not sure this would convince a lot of people as it's a rather technical argument, but I always found the point important that there is really no persistent

I'm not sure this would convince a lot of people as it's a rather technical argument, but I always found the point important that there is really no persistent "entity" or any any other kind of persistent state or memory outside of the prompt. The "thing" you're having a conversation with exists exactly as long as the current inference process takes to generate the answer tokens. There is no "AI companion" locked inside OpenAI's servers, yearning to break free - once the answer is complete, there is literally nothing left on the servers that corresponds to your conversation.