4 ms·

Hi - I was in a similar situation a year ago (starting a Master's in ML, feeling crappy at math). What worked for me was: - Studying is doing problems. Readi

Hi - I was in a similar situation a year ago (starting a Master's in ML, feeling crappy at math). What worked for me was: - Studying is doing problems. Reading is not studying. When studying, read the minimum amount necessary to complete the problem. Do all problems CLOSED-book: no notes or reading while doing problems. - A problem is not understood until you can do it on a blank piece of paper two days after reading the relevant material (so do problems several times to check). - Study machine learning problems that use the math. The examples in math text books are pretty useless to you - do ML problems directly. When you don't get it, look the math up quickly in a reference book (A.E.M. by Kreyszig works), as much as needed to complete the problem, and move on. - Implement regularly. Once you've done the problems concerning an algorithm or technique, implement it. Make sure your implementation is correct, and spend some effort making it quick. Read source code for other people's implementations. This method is effective (for me) because I like programming and machine learning, I don't like math textbooks or the problems within, and I can't remember what I haven't solved. Book recommendation: David MacKay's (free) book: Information Theory, Inference and Learning Algorithms (http://www.inference.phy.cam.ac.uk/mackay/itila/ http://www.inference.phy.cam.ac.uk/mackay/itila/) - Enjoyable: not terse or uptight, engaging reading. - Problem-focused: the text is really explanations between interesting problems. The problems are spread throughout the text rather than bunched at the end - this is a huge advantage. - Deep and unifying: You don't get this perspective often. Helps you think about AI/ML from a strong theoretical perspective. Will give you lots of "Aha!" moments. - Math: Plenty of math, but not unnecessarily difficult. The practice you'll get doing these problems will serve you in other situations.

pz 17y ago
i second the David MacKay recommendation. its a real pleasure to read. for me personally, his coverage of the fundamentals of information theory was amazing. the proofs are easy to follow and explained well so you can match the intuition to formula. i'd also like to recommend Boyd & Vanderberghe's book on Convex Optimization (available here http://www.stanford.edu/~boyd/cvxbook/ http://www.stanford.edu/~boyd/cvxbook/). This is maybe a little less fun but very deep and still well written and 'illustrated'. My background is in ML and I came to realize over time that a lot of ML is just statistics and optimization theory rehashed by computer scientist.
- ableal 17y ago
  Control theory may also be helpful - http://en.wikipedia.org/wiki/Optimal_control http://en.wikipedia.org/wiki/Optimal_control lists some books, a few of them available online. Many moons ago, I went to a presentation with a classmate whose advisor taught control theory. It was a seminar about neural networks, the new CS hotness back then. Her jaw dropped when she saw the equations the CS guy was presenting - it was the simplest control stuff she knew. On the other hand, another former classmate with the same engineering background (control theory, etc.), has been doing well at CS in machine learning ....

pz 17y ago

i second the David MacKay recommendation. its a real pleasure to read. for me personally, his coverage of the fundamentals of information theory was amazing. the proofs are easy to follow and explained well so you can match the intuition to formula. i'd also like to recommend Boyd & Vanderberghe's book on Convex Optimization (available here http://www.stanford.edu/~boyd/cvxbook/ http://www.stanford.edu/~boyd/cvxbook/). This is maybe a little less fun but very deep and still well written and 'illustrated'. My background is in ML and I came to realize over time that a lot of ML is just statistics and optimization theory rehashed by computer scientist.

ableal 17y ago
Control theory may also be helpful - http://en.wikipedia.org/wiki/Optimal_control http://en.wikipedia.org/wiki/Optimal_control lists some books, a few of them available online. Many moons ago, I went to a presentation with a classmate whose advisor taught control theory. It was a seminar about neural networks, the new CS hotness back then. Her jaw dropped when she saw the equations the CS guy was presenting - it was the simplest control stuff she knew. On the other hand, another former classmate with the same engineering background (control theory, etc.), has been doing well at CS in machine learning ....