The future is exciting and terrifying. Scaling laws have resulted in GPT-4 and they keep going. The design space for LLMs is bigger than just transformer arch

The future is exciting and terrifying. Scaling laws have resulted in GPT-4 and they keep going. The design space for LLMs is bigger than just transformer architectures. It includes long prompts, retrieval, chains, data quality, compression, etc, which potentially stack on top of HW acceleration. These are the worst models that we will see over the next decade. It’s going to be wild.