3 ms·
Markov Chains are analytical models, as opposed to LLMs. Big difference.
by jdthedisciple 4y ago
Markov Chains are analytical models, as opposed to LLMs.
Big difference.
- ftxbro 4y agoAn LLM (one that for example gives a probability distribution over tokens that follow a 2048 token context sequence) is a giant high-order Markov chain with an implicit transition matrix. I guess you are hung up on the fact that the full transition matrix of an LLM is never actually constructed in memory in closed form. Maybe according to your definition that makes it not an 'analytical model'.
- jdthedisciple 4y agobesides, where did you get the 2048 token number from?
- ftxbro 4y agoThe LLM GPT-3 for example has a 2048-token-long context.
- deleted 4y ago[deleted]