4 ms·
I skimmed through the book, and it's lacking the information theory foundations. For example, "trust region methods" come from maximizing the policy's relative
by programjames 3mo ago
I skimmed through the book, and it's lacking the information theory foundations. For example, "trust region methods" come from maximizing the policy's relative entropy (to a reference policy) under a tournament system where high-scoring agents are exponentially likely to survive. In general, a reward is the negative bits it costs an environment to propagate an agent (multiplied by some temperature).
- ainch 3mo agoDo you have a good source on this information theory framing? I don't remember it being covered in Sutton & Barto.
- porridgeraisin 3mo agoIt's just another way to frame it. It's as foundational as the many other ways to frame it. I'm not aware of any major insight you get specifically from this framing. Is there one?