3 ms·
> The bellman equations (exactly as written above) are not found in ML libraries. This is because they work assuming you know a model of the data. Most real wor
by in-silico 7mo ago
> The bellman equations (exactly as written above) are not found in ML libraries.
This is because they work assuming you know a model of the data. Most real world RL is model-free RL.
Q-learning (the usual application of the Bellman equation) is generally model-free. It is also commonly found in reinforcement learning libraries.
- porridgeraisin 7mo agoUsually deep Q learning is found in libraries where you function-approximate Q with a NN, which I alluded to in one of my later paragraphs (the approximation one).
- in-silico 7mo agoModel-free RL doesn't mean you aren't training a model. It means that you aren't explicitly building a model of the environment's f(s,a)=(s',r) transition function, which methods like Dreamer do. Q-learning only approximates the Q-value function, not the full state transition, so it is model-free.
- porridgeraisin 7mo agoI know that... I didn't say Q learning is not model-free. Do you seriously think a writer of that post would think model-free RL means "not training a model at all"? > Dreamer Especially when I later mentioned dreamer myself specifically as a model-based algorithm > Site guidelines: Please respond to the strongest plausible interpretation of what someone says