A more prosaic explanation of the birthday cake text is that the transformers architecture result in a system which returns learned information transformed, map

A more prosaic explanation of the birthday cake text is that the transformers architecture result in a system which returns learned information transformed, mapped according various context tokens. The system learns "X being the basis of civilization" type arguments and learns maps of those arguments according to various things including "from Y's point of view". One thing that I've noticed that ChatGPT can't do, not at all, is "write a show rather than tell paragraph". For example, writing a paragraph describing a man walking down the street in such way that some qualities (old, tired, etc) are evident without being directly stated (For example: "He stepped slowly and carefully unto the street, using his cane to steady himself" etc). Because the thing is ultimately transforming language. Oppositely, that these system can still do a lot is naturally an indication that a lot of what's being demonstrated are "just" structures in language - though if the structures are quite, the "just" part is quite relative.