That's interesting! But it does show that the hype around ChatGPT is misplaced: as impressive as it is (and I do find it impressive) it doesn't really build a "

That's interesting! But it does show that the hype around ChatGPT is misplaced: as impressive as it is (and I do find it impressive) it doesn't really build a "model" of the conversation; you have to help it all the way or it will go astray, since nonsensical board game moves make as much sense from a conversational point of view. It's also easy to make it hallucinate nonfactual information, which makes it bad at exploring things you're unsure about and about which you could inadvertently write misleading questions (Note this isn't the same as asking misleading questions to a human who will answer confidently out of arrogance; ChatGPT has no "ego" but it will write completely false/nonfactual answers if asked to by mistake. I have examples of this). It's easy to get confused about GPT's limitations because it's a pretty successful parrot, and it writes convincing conversations in a vast number of cases.