4 ms·

Sutton and Barto book implementation

80 points by ivanbelenky 1y ago

sage76 1y ago
Damn this is a lot of work. Bookmarked.
- deleted 1y ago
  [deleted]
- ivanbelenky 1y ago
  It has not been stress tested, or optimized, tread lightly and thanks a lot for appreciating the work.
mark_l_watson 1y ago
Very nice, thanks for doing this. I have experimented a lot with the "official" Common Lisp and Python examples for the Sutton/Barto RL book, and I will enjoy your implementations also! For reference, original examples in Lisp and Python: http://incompleteideas.net/book/code/code2nd.html http://incompleteideas.net/book/code/code2nd.html A bunch of implementations with all kinds of use cases (e.g., using OpenAI RL Gym, etc.): Here are some resources with code examples and implementations related to the Sutton and Barto "Reinforcement Learning: An Introduction" book: Code for Sutton & Barto Book: Reinforcement Learning: An Introduction: The official website for the book provides links to various software and re-implementations in different languages, including Python, Julia, and Lisp. This is a great starting point to find code directly associated with the book's examples and exercises. Link: http://incompleteideas.net/book/code/code2nd.html http://incompleteideas.net/book/code/code2nd.html jovsa/rl-examples-sutton-and-barto-book on GitHub: This repository offers Python implementations of examples from the book, organized by chapter. It includes code for figures and examples from various chapters, covering topics like Gridworld, Blackjack, and the Mountain Car task. Link: https://github.com/jovsa/rl-examples-sutton-and-barto-book https://github.com/jovsa/rl-examples-sutton-and-barto-book kamenbliznashki/sutton_barto on GitHub: This repository provides Python implementations of RL algorithms for the examples and figures in the Sutton and Barto book. It covers a wide range of topics from multi-armed bandits to policy gradient methods. Link: https://github.com/kamenbliznashki/sutton_barto https://github.com/kamenbliznashki/sutton_barto boldyshev/sutton on GitHub: This repository contains Python implementations of example experiments (figures) and programming exercises from the second edition of the book. Chapters are added as the author studies the book, making it a potentially growing resource. Link: https://github.com/boldyshev/sutton https://github.com/boldyshev/sutton AntonioSerrano/Implementation-of-RL-algorithms-from-Sutton-and-Barto-2018 on GitHub: This repository offers implementations in Python using OpenAI Gym and Tensorflow, covering exercises and solutions to complement the book and David Silver's RL course. It includes various algorithms like Dynamic Programming, Monte Carlo, Temporal Difference, and Policy Gradient methods. Link: https://github.com/AntonioSerrano/Implementation-of-RL-algorithms-from-Sutton-and-Barto-2018 https://github.com/AntonioSerrano/Implementation-of-RL-algor...
- ivanbelenky 1y ago
  my code is not as good as anything above most probably. Ive done this exploring while studying. No linter no typechecker, grug engineer mentality. But thanks nevertheless for the comment :)
  - mark_l_watson 1y ago
    well, it looks good to me.
mark_l_watson 1y ago
I want to add a second comment: Professors White & White (a husband and wife team) have a very good set of courses on RL on Coursera: https://www.coursera.org/specializations/reinforcement-learning https://www.coursera.org/specializations/reinforcement-learn...
- ivanbelenky 1y ago
  Lovely!
AndrewKemendo 1y ago
Let me know if anyone fills out the true online Sarsa section with a working example in a robot
mtamana 1y ago
[flagged]
vlad 1y ago
The authors were professor and grad student at UMass Amherst, and are the current winners of the Turing Award. https://www.cics.umass.edu/ https://www.cics.umass.edu/ https://www.nsf.gov/news/ai-pioneers-andrew-barto-richard-sutton-win-2024-turing https://www.nsf.gov/news/ai-pioneers-andrew-barto-richard-su...
ultrasounder 1y ago
Super helpful while I come upto speed with this field in general. Currently taking the XCS234(RL @ Stanford online) and this book is referenced for everything.

80 points by ivanbelenky 1y ago

sage76 1y ago

Damn this is a lot of work. Bookmarked.

deleted 1y ago
[deleted]
ivanbelenky 1y ago
It has not been stress tested, or optimized, tread lightly and thanks a lot for appreciating the work.

mark_l_watson 1y ago

Very nice, thanks for doing this. I have experimented a lot with the "official" Common Lisp and Python examples for the Sutton/Barto RL book, and I will enjoy your implementations also! For reference, original examples in Lisp and Python: http://incompleteideas.net/book/code/code2nd.html http://incompleteideas.net/book/code/code2nd.html A bunch of implementations with all kinds of use cases (e.g., using OpenAI RL Gym, etc.): Here are some resources with code examples and implementations related to the Sutton and Barto "Reinforcement Learning: An Introduction" book: Code for Sutton & Barto Book: Reinforcement Learning: An Introduction: The official website for the book provides links to various software and re-implementations in different languages, including Python, Julia, and Lisp. This is a great starting point to find code directly associated with the book's examples and exercises. Link: http://incompleteideas.net/book/code/code2nd.html http://incompleteideas.net/book/code/code2nd.html jovsa/rl-examples-sutton-and-barto-book on GitHub: This repository offers Python implementations of examples from the book, organized by chapter. It includes code for figures and examples from various chapters, covering topics like Gridworld, Blackjack, and the Mountain Car task. Link: https://github.com/jovsa/rl-examples-sutton-and-barto-book https://github.com/jovsa/rl-examples-sutton-and-barto-book kamenbliznashki/sutton_barto on GitHub: This repository provides Python implementations of RL algorithms for the examples and figures in the Sutton and Barto book. It covers a wide range of topics from multi-armed bandits to policy gradient methods. Link: https://github.com/kamenbliznashki/sutton_barto https://github.com/kamenbliznashki/sutton_barto boldyshev/sutton on GitHub: This repository contains Python implementations of example experiments (figures) and programming exercises from the second edition of the book. Chapters are added as the author studies the book, making it a potentially growing resource. Link: https://github.com/boldyshev/sutton https://github.com/boldyshev/sutton AntonioSerrano/Implementation-of-RL-algorithms-from-Sutton-and-Barto-2018 on GitHub: This repository offers implementations in Python using OpenAI Gym and Tensorflow, covering exercises and solutions to complement the book and David Silver's RL course. It includes various algorithms like Dynamic Programming, Monte Carlo, Temporal Difference, and Policy Gradient methods. Link: https://github.com/AntonioSerrano/Implementation-of-RL-algorithms-from-Sutton-and-Barto-2018 https://github.com/AntonioSerrano/Implementation-of-RL-algor...

ivanbelenky 1y ago
my code is not as good as anything above most probably. Ive done this exploring while studying. No linter no typechecker, grug engineer mentality. But thanks nevertheless for the comment :)
- mark_l_watson 1y ago
  well, it looks good to me.

mark_l_watson 1y ago

I want to add a second comment: Professors White & White (a husband and wife team) have a very good set of courses on RL on Coursera: https://www.coursera.org/specializations/reinforcement-learning https://www.coursera.org/specializations/reinforcement-learn...

ivanbelenky 1y ago
Lovely!

AndrewKemendo 1y ago

Let me know if anyone fills out the true online Sarsa section with a working example in a robot

mtamana 1y ago

[flagged]

vlad 1y ago

The authors were professor and grad student at UMass Amherst, and are the current winners of the Turing Award. https://www.cics.umass.edu/ https://www.cics.umass.edu/ https://www.nsf.gov/news/ai-pioneers-andrew-barto-richard-sutton-win-2024-turing https://www.nsf.gov/news/ai-pioneers-andrew-barto-richard-su...

ultrasounder 1y ago

Super helpful while I come upto speed with this field in general. Currently taking the XCS234(RL @ Stanford online) and this book is referenced for everything.