3 ms·
> Calling an LLM a "Markov model" reveals an agenda rather than an insight. What agenda? Knowing that LLM's are markov chains helps me understand and makes me
by Jensson 2y ago
> Calling an LLM a "Markov model" reveals an agenda rather than an insight.
What agenda? Knowing that LLM's are markov chains helps me understand and makes me better at using them, since I know every word is just calculated probabilistically based on the previous N words, the fastest way to describe that is to say it is a Markov chain. This makes it hard for them to coordinate their early words with things that will happen later, all that from just knowing it is a Markov chain.
I feel the knee jerk rejection of this statement is more of an agenda, why do you feel it is so wrong to say it is a Markov chain? Are you sure there is no agenda behind those feelings?