4 ms·
Do you have a good textbook to reference to where the theory starts from PAC-Bayes?
by 5555watch 20d ago
Do you have a good textbook to reference to where the theory starts from PAC-Bayes?
- srean 20d agoYou can try this one https://books.google.co.in/books/about/User_friendly_Introduction_to_PAC_Bayes.html https://books.google.co.in/books/about/User_friendly_Introdu... Free download here https://arxiv.org/abs/2110.11216 https://arxiv.org/abs/2110.11216 One of the ICMLs had a nice tutorial by Langford and Banerjee on the relationship between the different style of bounds. 2003, I think.
- hodgehog11 20d agoJust to add on top of the quality reference provided by srean, I like to first drill in Bayesian principles and then use this article to derive PAC-Bayes from that: https://arxiv.org/abs/1605.08636 https://arxiv.org/abs/1605.08636 Regular PAC falls out by taking a uniform prior over a finite hypothesis class (and then building up VC dimension if desired, but usually by this point you realise why the bounds are unlikely to be good).
- srean 20d agoSeems I was misremembering the dates. The Langford and Banerjee papers/turorials I had in mind were On Bayesian Bounds https://dl.acm.org/doi/10.1145/1143844.1143855 https://dl.acm.org/doi/10.1145/1143844.1143855 Tutorial on Practical Prediction Theory for Classification https://jmlr.csail.mit.edu/papers/v6/langford05a.html https://jmlr.csail.mit.edu/papers/v6/langford05a.html The first one is quite in the same spirit that you like.