It feels very strange to put "next token prediction" on the same level as "E = mc2". "next token prediction" is an experiment that's puzzling, while E=mc2 is a

It feels very strange to put "next token prediction" on the same level as "E = mc2". "next token prediction" is an experiment that's puzzling, while E=mc2 is a theory (that Einstein developed to explain the puzzling experiments that were unexplained at the time). It may or may not be that in the end there is a "simple formula" that explains why LLMs work, but we're not there yet.