4 ms·
What about a language model requires knowledge of previous states to choose the next state based on the current state? Are you asserting that only the last word
by cestith 1y ago
What about a language model requires knowledge of previous states to choose the next state based on the current state? Are you asserting that only the last word chosen is the current state? Couldn’t the list of symbols chosen so far be the current state, so long as no data is carried forward about how that list has been generated?
- coliveira 1y agoYou're right, the recent memory of the LLM is the state, that's why it needs to be so deep to be effective compared to a traditional MC.