> I brought up the metaphor of the Wizard of Oz. In the movie, the wizard is posing as an immensely powerful entity but turns out to just be a guy operating mac

> I brought up the metaphor of the Wizard of Oz. In the movie, the wizard is posing as an immensely powerful entity but turns out to just be a guy operating machinery. The drum I keep beating: Every AI assistant you "talk to" is a fictional character in a movie-script. There's code that "acts out" its lines to you, and your own text becomes "then the user said X" lines. The nameless ego-less LLM iteratively Makes The Document Longer, in ways that fit all the documents its seen. An ultra-powered mad-libs. > For instance, I told it to become “Robert,” who talks only in dumb ways Right: The exact same system could have the human user conversing with Dracula, Vampire Lord of the Night. "Dracula" would be demanding that you convince all your friends to donate blood and boycott products made with garlic. The character would "fear" wooden stakes and crucifixes--in the same way that this "AI" character doesn't "want" to be turned-off or deleted. However making software that writes stories about X is not the same as creating X... not even when X happens to be "an AI sharing the same product name." > I told Bob that he is not alone: some of the smartest people I know are getting fooled. Not only that, but the mistake is being deliberately encouraged by other humans. They have deliberately built the system make users think they are talking to a discrete intelligent entity... and for investors to believe they have invented the same.