Something that deeply frustrates me, as someone who did R&D on model architectures, is how similar the modern LLM model architectures are to GPT2. (This is a b

Something that deeply frustrates me, as someone who did R&D on model architectures, is how similar the modern LLM model architectures are to GPT2. (This is a bit disingenuous, as lots/most of work is spent on the scaling and training side of things.)