5 ms·
Reinforcement Learning: An Introduction (2018)
- dang 2y agoRelated. Others? Reinforcement Learning: An Introduction (2018) [pdf] - https://news.ycombinator.com/item?id=19191746 https://news.ycombinator.com/item?id=19191746 - Feb 2019 (23 comments) Reinforcement Learning: An Introduction, Second Edition - https://news.ycombinator.com/item?id=18547998 https://news.ycombinator.com/item?id=18547998 - Nov 2018 (6 comments) New Draft of “Reinforcement Learning: An Introduction, Second Edition” - https://news.ycombinator.com/item?id=12568414 https://news.ycombinator.com/item?id=12568414 - Sept 2016 (33 comments) Reinforcement Learning: An Introduction - https://news.ycombinator.com/item?id=1083662 https://news.ycombinator.com/item?id=1083662 - Jan 2010 (4 comments)
- whatever1 2y agoDo we have any real world applications where the policy satisfies strictly some constraints (think physics: conservation of mass etc)? There is research in the field but not sure if anything is in production.
- amelius 2y agoThis is what I'd like to know as well.
- cg30e 2y agoWhat are the math prerequisites for this book? What other reinforcement learning books or courses are recommended for beginners?
- kasmura 2y agoThis is THE book. It is suitable for beginners with basic math background equivalent to CS undergrad I’d say
- textlapse 2y agoBasic probability/stats and some basic fundamentals around dynamic programming/recursion would be very helpful. The big problem I found with this field is that the core ideas are very subtly built on top of each other. Without a proper teacher or an environment to study, self-study is much much harder. (Past chapter 5, it should be a breeze as the foundation would have been strongly set)
- deleted 2y ago[deleted]
- kleiba 2y agoInteresting to read the last section of the last chapter (17.6 Reinforcement Learning and the Future of Artificial Intelligence) given that the book is from 2020 and ChatGPT (in which RL plays a key role) was published in 2022.
- byyoung3 2y agoi pretty much have read this book. pretty boring tbh but still good. i would recommend doing hands on implementations with the cartpole environment
- gozzoo 2y agoThis book seems to be very theoretical. Can someone recoment more practical books with code samples using some modern ML framework, probably something like _Hands-On Machine Learning_ by Geron Aurelien
- Jagerbizzle 2y agohttps://course.fast.ai/ https://course.fast.ai/ "You’ll see that fast.ai’s way of teaching is very different to what you might be used to, if you did a technical degree at university. Nearly all technical subjects at university are taught “bottom up”: start with basic foundations, and gradually work up to complete useful solutions to real world problems. But we go “top down”: start with complete useful solutions to real world problems, and gradually work down to the basic foundations. Education experts recommend this approach for more effective learning."
- jhanschoo 2y agoI don't think this discusses reinforcement learning (not sure about the comment you responded to). To my recollection, the first course goes through basic architectures and concepts, and the second discusses autoenvoders.
- intelkishan 2y agoYeah, the Fast.ai course does not discuss RL. The first part focuses on DL and the second part focuses on Diffusion based models
- pkoperek 2y agoTo be honest I think this may be actually an advantage: it explains concepts which otherwise are just weird parameters in code. Since it is pretty lengthy I would actually recommend to read the chapters relevant to a specific method you are interested in (maybe going a backwards to build the right context). If you I'd combine it with e.g. https://spinningup.openai.com/en/latest/ https://spinningup.openai.com/en/latest/ or doing some toy projects with https://stable-baselines3.readthedocs.io/en/master/ https://stable-baselines3.readthedocs.io/en/master/ it would probably render the most value.
- kengoa 2y agoThere's lecture notes on this book available from David Silver at UCL: https://www.davidsilver.uk/teaching/ https://www.davidsilver.uk/teaching/
- malshe 2y agoThanks for sharing this. David Silver's YouTube lectures on RL are great too
- dlevine 2y agoI did the Reinforcement learning specialization on Coursera (https://www.coursera.org/specializations/reinforcement-learning https://www.coursera.org/specializations/reinforcement-learn...), which used this book. The specialization was great, but I would say that I had a lot of trouble following this book. I would find that there was a concept I didn't quite get, and then other things would build on top of that concept. I ended up feeling like I didn't quite grasp everything, even though I read most of the chapters more than once. I do feel like there are a lot of things that aren't fully explained in this book, and maybe it is expected that the reader has some prior knowledge that I didn't have. For example, they never really explain the basic notation for backup diagrams. They just show one and mention that they are a thing. The mathematical notation they use also isn't really explained - they just start showing formulas of increasing complexity. It's possible to look these things up elsewhere, but they probably could have spent just a little bit more time explaining some of the basics and made it easier to follow.
- cg30e 2y agoWhat made the specialization great? Did they explain things well where the book didn’t? Are the courses self-contained? What are the prerequisites?
- dlevine 2y agoThe specialization did a good job of explaining all of the basic concepts of RL in a fairly understandable way. The instructors don't assume a huge amount of prior knowledge, and make it pretty accessible. They split reasonably between theory and applications. I would say that you need some background in math and ML. The courses do build upon each other. You could do just the first course and get a good overview of RL without diving into the details, but I don't think you could start midway through the specialization. I would also suggest looking at the Hugging Face Deep Reinforcement Learning course. That one is very different (focused more on application than diving deep into the theory), but it's taught by a non-academic and really tries to explain the concepts in a way that is approachable to most programmers.
- nivertech 2y ago
- smokel 2y agoThe field of reinforcement learning is huge. There are many different algorithms with catchy names, which are mostly variations of two basic algorithms. However, each variation can easily keep you busy for a PhD or two. The book referenced here (the "Sutton and Barto book") covers the basics. If you like more, or more recent texts, consider these (mostly available for free): Multi-Agent Reinforcement Learning: Foundations and Modern Approaches, 2024, https://www.marl-book.com/ https://www.marl-book.com/ Distributional Reinforcement Learning, 2023, https://www.distributional-rl.org/ https://www.distributional-rl.org/ Deep Reinforcement Learning, 2022, https://deep-reinforcement-learning.net/ https://deep-reinforcement-learning.net/
- deleted 2y ago[deleted]