2 ms·
How do you apply Q-learning to LLMs? They would need to estimate the action-value of each action in a given state. How can you pick the best action when you onl
by visarga 3y ago
How do you apply Q-learning to LLMs? They would need to estimate the action-value of each action in a given state. How can you pick the best action when you only have a scoring function that would rate how good each action is, but the action space is infinite. You can't pick the best action.
It is probably something else, like Decision Transformer.
- silveraxe93 3y agoThat got solved using Deep-Q learning. I think David Silver did loads of work on it? Basically, instead of computing every state value like in normal Q learning. You use a Neural Network to estimate the best value.
- visarga 3y agoIf I remember correctly DQLearning is using discrete action space, limited number of possible actions which makes it possible to simply take the argmax_i(qvalue(state, action_i))