Language model != Transformers. The scaling power of the transformer architecture allows LLMs to exist, but LMs are just one application of transformers. The ar

Language model != Transformers. The scaling power of the transformer architecture allows LLMs to exist, but LMs are just one application of transformers. The architecture itself is extremely generic and is applicable to nearly every problem. I'm pretty certain whatever comes after GPT-4 will still be using layered self-attention.