How inference works is you do a bunch of math and token weights come out, and then you pick one based on some sampling policy. “Prediction” is really about how

How inference works is you do a bunch of math and token weights come out, and then you pick one based on some sampling policy. “Prediction” is really about how they’re trained than how they run afterwards.