4 ms·
LLMs are Markov chains in latent space, it's the latent representation that give them their power, but ultimately there's not as much difference as one would su
by PheonixPharts 3y ago
LLMs are Markov chains in latent space, it's the latent representation that give them their power, but ultimately there's not as much difference as one would suspect.
- esafak 3y agoThey're different because Markov models are stateless whereas LLMs are stateful. https://en.wikipedia.org/wiki/Markov_property https://en.wikipedia.org/wiki/Markov_property
- Jensson 3y agoCurrent LLMs are stateless as far as we know, their state when computing a new token is only the preceding text tokens, they don't store any metadata or save state from the previous calculations.