> So a perfect world model is enough to win at reinforcement learning. Can you show that if you maximized reward in some RL problem, it means you necessarily bu

> So a perfect world model is enough to win at reinforcement learning. Can you show that if you maximized reward in some RL problem, it means you necessarily built a perfect world-model? No? Because maximizing reward for a specific problem may mean avoiding some states entirely, so your model has no need of understanding transitions out of those states, only how to avoid landing in them. E.g. if you have a medical application about interventions to improve outcomes for patients with disease X, it's unnecessary to refine the part of your model which would predict how fast a patient would die after you administer a toxic dose of A followed by a toxic dose of B. Your model only need to know that administering a toxic dose of A always leads to lower value states than some other action. I think a "perfect" world model is required by a "universal" AI in the sense that the range of problems it can handle must be solved by optimal policies which together "cover" all state transitions (in some universe of states).