4 ms·
Thanks for this great contribution. I would like to be able to read the math in DL papers. (sorry I'm asking for something that it's too broad) 1) How much d
by sydl 9y ago
Thanks for this great contribution.
I would like to be able to read the math in DL papers.
(sorry I'm asking for something that it's too broad)
1) How much does this document cover the notations in those papers.
2) When I read a paper and if I am not sure what the math means, does that mean that I did not grok the subject yet, or the math presented in that paper goes beyond the math given in this Matrix Calculus document (assuming I studied well this document).
- pacbard 9y agoI would say try to get as much of the intuition behind the math as you can. Knowing what an equation means rather than fully understanding the Greek notation is what it matters for practice. If you want to substantially contribute to the theoretical CS literature, you will need to have a good handle of the notation (for obvious reasons). Note: I come from math and Econ, so the split between practitioner and theorists might be different for CS/ML.
- blt 9y agoWhile matrix derivatives are important, there is also a lot of other math in DL papers. In particular, a lot of the probability side concerns expectations, KL divergences, entropy, etc., which are all defined in terms of integrals or sums. You need undergraduate-level probability background.
- jph00 9y agoThe first 5 chapters of the Goodfellow deep learning book are a great resource for understanding the probability, linear algebra, optimization, and information theory you need to digest deep learning papers.