4 ms·
> But a Markov chain large enough to act like an LLM would collapse into a black hole You mean a full state table Markov Chain? Markov chains doesn't need to h
by Jensson 2y ago
> But a Markov chain large enough to act like an LLM would collapse into a black hole
You mean a full state table Markov Chain? Markov chains doesn't need to have all states in a table, it just needs to be a function, a table is one implementation of a function but anything can do.
- ben_w 2y agoIndeed. Although my understanding is that under quantum mechanics it's necessary to set the probability of all transitions between states to a number larger than zero, I also understand that for practical purposes the probability for the overwhelming majority of transitions are close enough to zero as to require fancy notation (at the very least, nested exponentiation) to express quite how unlikely they are.