There's another illusory effect here: Humans are being encouraged to confuse a fictional character with the real-world "author" system. I can create a mad-libs

There's another illusory effect here: Humans are being encouraged to confuse a fictional character with the real-world "author" system. I can create a mad-libs program which dynamically reassembles stories involving a kind and compassionate Santa Claus, but that does not mean the program shares those qualities. I have not digitally reified the spirit of Christmas, not even if excited human kids contribute some of the words that shape its direction and clap with glee. P.S.: This "LLM just makes document bigger" framing is also very useful understanding how prompt injection and hallucinations are constant core behaviors, which we just ignore except when they inconvenience us The assistant-bot in the story can be twisted or vanish so abruptly because it's just something in a digital daydream.