5 ms·
I love textbooks and spend more of my childrens' inheritance on them than I should. But what MOOCs give me is the exercises. I often think I understand a prob
by rfrey 12y ago
I love textbooks and spend more of my childrens' inheritance on them than I should.
But what MOOCs give me is the exercises. I often think I understand a problem, but it's only after getting 2.1/10 on a Coursera quiz that I realize I've missed a key step or concept.
Many textbooks have exercises but few have solutions. I've been working through Barto and Sutton's Reinforcement Learning for example (again), and although I do the exercises and programming questions, I never know if I've gotten it right. My experience with MOOCs shows I probably haven't in a large number of cases.
The best of both worlds is when I can follow a MOOC with the textbook to gain more depth, for example with the PGM course on Coursera.
- grayclhn 12y agoTextbooks also have "examples" which are exercises with worked answers. If you cover up the solution and try to work through it on your own first, a worked example is very instructive. It's just as instructive (and usually pretty embarrassing) to go back and work through the examples a day or two after you first read them. And then a day or two after that :)
- ced 12y agoIncidentally, I liked Barto and Sutton's book, but it feels a bit dated. Does anyone have another reinforcement learning book to recommend?
- camlinke 12y agoI chatted with Rich Sutton recently and he said he was writing and update - not sure how far along in the process he is though. Also Csaba Szepesvári (a colleague of Rich's at the U of A) has a free RL book you can download. http://www.ualberta.ca/~szepesva/RLBook.html http://www.ualberta.ca/~szepesva/RLBook.html
- noelwelsh 12y agoCsaba's book is the most up-to-date on RL I know of. Sutton and Barto is very old by now. For the POMDP side of things there are no recent books I know of, but http://www.cs.mcgill.ca/~jpineau/files/sross-jair08.pdf http://www.cs.mcgill.ca/~jpineau/files/sross-jair08.pdf is a recent enough survey. A related book is "Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems" which you can find at http://www.princeton.edu/~sbubeck/index.html http://www.princeton.edu/~sbubeck/index.html The bandit problem is very strongly related to the reinforcement learning problem, so you'll get some mileage out of studying bandits. Be aware this area is very maths heavy, which is good or bad depending on your background. If you like you like this stuff, also checkout "Prediction, Learning, and Games" which deals more with the "adversarial" setup.
- rfrey 12y agoAgree with others that Barto & Sutton is dated, but it's not obsolete. I was chatting a couple weeks ago with one of Sutton's former students, who has since Made Good in both academia and industry working on AI, and he said that the B&S book is all you need to be able to read the literature, so it's still a good place to start. Apparently Sutton was going to do a second book, but has since retreated to a second edition. I'll take it though. :) The Barto and Sutton book is available online too, http://webdocs.cs.ualberta.ca/~sutton/book/the-book.html http://webdocs.cs.ualberta.ca/~sutton/book/the-book.html although the printed copy is a high-quality publication so maybe worth it.
- achompas 12y agoAgreed, it's also a bit topical. Check out Wiering + van Otterlo's "Reinforcement Learning: State-of-the-art." Covers many new techniques--I used it as a reference for a project earlier this year: http://www.springer.com/engineering/computational+intelligence+and+complexity/book/978-3-642-27644-6 http://www.springer.com/engineering/computational+intelligen...
- mathattack 12y agoYes indeed. It's the exercises and feedback that help a lot. I believe that the optimized future is MOOC Problems to start, Texts and Recorded lectures to help where you struggle, humans to help when the texts and recording miss your mistakes.