Why is it so difficult to conceive that what these models produce is novel, and capable of high level technical and semantic correctness? Transformer models ar

Why is it so difficult to conceive that what these models produce is novel, and capable of high level technical and semantic correctness? Transformer models aren't auto-complete engines. They're semantic graphs describing the probabilities of token positions within a sequence of a few thousand tokens. They go deep, with a nuanced and hierarchical representation of concepts. The representation is functional, conditionally altering the behavior of tokens within its sequence context. Because of the hierarchical nature of the internal rules, when the model "learns" things, it extracts meta-patterns from the training data, and can chain a seemingly arbitrary number of such rules together to produce an output. Auto-complete systems generally fail at the first level of abstraction. If transformers have sufficient vocabulary size and high quality data, they're capable of learning any degree of abstractions represented within the training corpus. They're not magic, or intelligent like human brains, but they're allowing programmers to model very complex systems and produce novel material. This ranges from constructed languages to the rules of programming to storytelling and poetry to protein sequencing and more. You should be in awe of gpt-3, or at the very least, far less dismissive. It represents a quantum leap in the power of software, and we're barely scratching the surface of what's possible. Recurrence and episodal memory, along with Moore's law and/or a substantial decrease in the size of the models, could be the thing that achieves human level agi that lives on consumer grade devices.