You don't see the intermediate steps from a 10x10 neural net to LLMs? Like, a whole decade of ML: better optimisers, better init, residual connections, tokenis

You don't see the intermediate steps from a 10x10 neural net to LLMs? Like, a whole decade of ML: better optimisers, better init, residual connections, tokenisation and token embeddings, training with large batches over thousands of machines, the attention mechanism, causal masking, flash attention and other memory optimisations, and even having the foresight to train on the totality of web text. Not seeing the intermediate steps doesn't mean they are not essential and needed. If you still believe a toy 10x10 fully connected net is the same with current models (bar scaling), then what is you opinion on MLP-Mixer? That was an "MLP is all you need" moment but it didn't lead to adoption.