3 ms·
Well, on Wikipedia you can read "Random process independent of past history" and "a stochastic model describing a sequence of possible events in which the proba
by GaggiX 3y ago
Well, on Wikipedia you can read "Random process independent of past history" and "a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event." honestly I would not consider the entire context window to be part of a single event, since the model generates tokens one at a time, separately.
- ftxbro 3y ago> honestly I would not consider the entire context window to be part of a single event, since the model generates tokens one at a time, separately. OK but traditional small language model Markov chains have context windows too, and they also generate tokens one at a time, separately. Maybe you are arguing that there is a qualitative difference between an eight token context window and an eight thousand token context window. I guess that would be fair enough.