3 ms·
I don't understand why the author thinks that the PAC learning framework is not useful. PAC gives quick back-of-napkin math for estimating how long it might tak
by lidHanteyk 7y ago
I don't understand why the author thinks that the PAC learning framework is not useful. PAC gives quick back-of-napkin math for estimating how long it might take to train a network, what kind of quality level is acceptable for inputs, etc.
- mlthoughts2018 7y agoPAC does not provide these types of estimates in most practical situations.
- 317070 7y agoCorrect me if I'm wrong, but doesn't PAC on real data usually just tells you that "it doesn't know the answer"? Like, it says the equivalent to: "I think this is the answer, but it might be miles off and I give it 50% it is even more than miles off." I found that it is able to give hard guarantees, by being very loose on the bounds. Do you have a real-world example where it actually gave something useful? I'm genuine curious, because I discarded it early on, but am willing to revisit it.
- eugenhotaj 7y agoI disagree, PAC bounds are usually way too loose to be useful in practice when it comes to NNs. I think part of the reason is that the way we measure model complexity for NNs does not correspond very well with their ability to generalize, as we've seen with recent papers like deep double descent [1]. Anecdotally, I was of the main authors of the AdaNet framework [2] where we used (approximate) Rademacher complexity (closely related to PAC) to bound the complexity of our learned ensemble models. We never got good results using the complexity measure and would basically turn it off for any practical problem we solved. [1]: https://openai.com/blog/deep-double-descent https://openai.com/blog/deep-double-descent [2]: https://github.com/tensorflow/adanet https://github.com/tensorflow/adanet