3 ms·
That is RNNs. LLMs (usually transformers) are non-markovian.
by visarga 2y ago
That is RNNs. LLMs (usually transformers) are non-markovian.
- Jensson 2y ago> That is RNNs. LLMs (usually transformers) are non-markovian. How? They have a hard cut off context, and randomly generates next state from that. That is the definition of a markov chain. It is a markov chain with a pretty large state at every point, but still a markov chain.