3 ms·
Markov chains are a one dimensional chain of immediate previous states and base their prediction on that very specific linear chain of states. A good way to im
by AnotherGoodName 2y ago
Markov chains are a one dimensional chain of immediate previous states and base their prediction on that very specific linear chain of states.
A good way to immediately grasp the flaws of a Markov chain is to imagine it predicting a pixel in a 2D image.
Sure in a 1D chain of states that goes something like [red green blue] repeatedly a Markov chain is a great predictor. But imagine a 2D image where the pattern is purely in the vertical elements but you’re passing in elements left to right then next row. It’s going to try to make a prediction based on the recent horizontal elements and there no good way to have context of the pixel above. A Markov chain doesn’t mix two states, its context is the immediate previous chain of states. It’s really really hard for a Markov chain to take the pixel above into context if your feeding it pixels left to right (same is true for words but the pixel analogy is easier to grasp).
Now you may think a good solution to this is to have a Markov chain have some mechanism to take multiple previous states from multiple contexts (in this case vertical and horizontal) and somehow weight each one to get the next state. Go down this path and you essentially go down the path of early neural networks.
- vrighter 2y agothe way I see it is that an llm esis* a markov chain. the only difference is a very long context, and a lossily compressed state transition "table".
- deleted 2y ago[deleted]
- AnotherGoodName 2y agoBut how to build a Markov chain that does what an LLM does? I've actually thought of this from time to time and come up with things like 'skip' states that just don't change the context on meaningless words, I've thought of maintaining distant states in some type of context on the markov chain, multiple contexts in multiple directions mixed with a neural network at the end (as per the 2D image example), etc. But then the big issue is in building the Markov state network. The attention mechanism and subsequent multi-layer neural network it uses is honestly extremely inelegant. Basically a entire datacenter sledgehammer of contexts and back-propagated neural networks to make something that works while Markov chains would easily run on an Apple 2 if you got it right (it's very fast to linearly navigate a chain of states). But the hard part is making Markov chains that can represent language well. The best we've done is very simple linear next letter predictors. I do honestly think there may be some value in stealing word weights from an attention mechanism and making Markov chains out of each of them. So each word starts navigating a markov chain of nearby words to end up at new states. You'd still mix all the contexts at the end with a neural network but it skips the computationally expensive attention mechanism. Still a hard problem to build these Markov chains sensibly though. Deepseek has shown us there's huge opportunities in optimizing the attention mechanism and Markov chains are super computationally simple but the 'how do you build good Markov chains' is a very hard problem.
- deleted 2y ago[deleted]
- globalnode 2y agoCould it be that the action of a Markov Chain on vectors, where the vector converges to an eigenvector, is similar to the action of an LLM under gradient descent, where vectors converge to a minimum? There's also a link between eigenvectors and solutions to simultaneous differential equations... I'm not that knowledgeable about all of this but it feels like there are many similarities.