I don't think a prediction is truly a prediction when it's not being compared against a reference. It's really only a prediction during training; the rest of th

I don't think a prediction is truly a prediction when it's not being compared against a reference. It's really only a prediction during training; the rest of the time it's synthesis. But again I'll repeat my point: "I also wouldn't underestimate the power of token prediction". It's very well possible that accurate token prediction may be the only necessary fundamental ingredient to weather forecasting, writing a successful novel, compiling a pitch deck for investors, designing a chemical plant... Humans can eat, talk, predict, reproduce, and wiggle our limbs and fingers, but it turns out that there's a lot of complex recipes that you can bake with those ingredients.