It's not wrong to say current LLM capability were indeed unimaginable back then. A lot about the transformer model was driven by experimentation, there was no u

It's not wrong to say current LLM capability were indeed unimaginable back then. A lot about the transformer model was driven by experimentation, there was no underlying theory per se that said we'd get as good results as we did. Everyone and the researchers themselves were shocked iirc; all this higher order reasoning emerged from being a next token predictor, yes that was the outlandish part.