6 ms·

Ask HN: 'Crash Courses' for Mathematics Related to DL, ML and Data Analysis

103 points by mayankkaizen 9y ago

I am specifically looking for free pdf or online materials for mathematics needed in ML, DL and Data analysis which doesn't necessarily go in depth. My primary aim to have a good top level view and if possible get hold of the most basics stuffs as soon as possible.

bjourne 9y ago
Read these books! * http://www-bcf.usc.edu/~gareth/ISL/ISLR%20First%20Printing.pdf http://www-bcf.usc.edu/~gareth/ISL/ISLR%20First%20Printing.p... * http://web4.cs.ucl.ac.uk/staff/s.prince/book/book.pdf http://web4.cs.ucl.ac.uk/staff/s.prince/book/book.pdf * http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%... I am not an expert in ML. I have heard experts in ML call the above books introductory.
abhinavkits501 9y ago
https://developers.google.com/machine-learning/crash-course/ml-intro https://developers.google.com/machine-learning/crash-course/...
BiancaDelRio 9y ago
http://www.cs.cornell.edu/jeh/book.pdf http://www.cs.cornell.edu/jeh/book.pdf
- mayankkaizen 9y ago
  Thanks!
- WalterGR 9y ago
  Foundations of Data Science Avrim Blum, John Hopcroft, and Ravindran Kannan Copyright 2015
dsacco 9y ago
I’m sorry to say this because I observe that we get these questions somewhat frequently, but I don’t think what you’re asking for - in the format, goals and speed you’d like it - is a thing. That’s a pretty discouraging statement, so let me try to be more helpful. The mathematics comprising contemporary machine learning and data analysis primarily consists of linear algebra, calculus, mathematical statistics and, at the high end, probability theory. There are boutique efforts to use things like topology but that’s non-standard. As it stands, your question is underspecified, which is what I repeat for most of these questions. The answers you receive here are going to variously interpret what you’re actually looking for and prescribe based on the author’s interpretation and intuition. It would be more useful if you explained exactly what your goals are. Here’s precisely the problem: you apparently don’t know these mathematics already, but you’re asking for something that “doesn’t necessarily go into depth.” How much depth is too much, and how would you know that exactly given your present unknown unknowns? If you explain what you actually want to achieve, we might be able to 1. Tell you that you don’t actually need that “top level view” to achieve what you’d like, 2. Tell you that such a top level view is not nearly enough for what you’d like to do, or 3. Tell you that a top level view is coherent, and optimize the best materials for you to learn from based on what you want to do. It would also help us make recommendations if you explained what your current level of mathematical background looks like. Have you taken linear algebra at least once? Exactly how basic do our resources have to be? Different textbooks written for different audiences can variously explain the same concept in two pages or 10 pages, and they can emphasize different things. I’d like to help, and I probably can, but it would go a long way if you could tell us what your end goal is instead of what you interpret as the next step towards that goal. Then we can provide resources based on your mathematical maturity.
- jxub 9y ago
  Topology for ML sounds amazing. Got any papers/links?
  - dsacco 9y ago
    Sure, try this: http://math.ucr.edu/home/baez/information/ http://math.ucr.edu/home/baez/information/. In a nutshell, you take probability distributions and project them onto topological manifolds, such that the distribution consists of points on the manifold. You can find more by searching for key words like "information geometry" or "differential geometry machine learning".
    - mlevental 9y ago
      there's also the stuff that ayasdi and carlsson were/are doing but i never really saw the point of that (e.g. just compute connected components instead persistent homology).
- mayankkaizen 9y ago
  OP here. Thanks for taking time to reply. Really thank you. Obviously I know very little about these areas and that is why I posted such a question. By posing this question, I didn't mean that I wanted to 'get there' quickly. In fact I am in no hurry and willing to go as deep as possible. What I actually mean to ask is that I was looking for a kind of syllabus/crash courses so that I can get an idea about what kind of mathematics is involved in these areas. Based on that, I'll be able to start my studies in much more organized way. Also, from there on, I myself be able to figure out how to look for further resources. This is the style of learning I usually follow. Hope you understand my point.
sdiq 9y ago
First, please read and internalize dsacco's comment. In the meantime, the following might help. But from experience, if your math is already shaky, you would certainly need to look for resources covering everything listed in a more in depth manner. Basic calculus refresher - http://www.stat.wisc.edu/~ifischer/calculus.pdf http://www.stat.wisc.edu/~ifischer/calculus.pdf Learning from Data - http://www.inf.ed.ac.uk/teaching/courses/fmcs1/readings/matlab-barber.pdf http://www.inf.ed.ac.uk/teaching/courses/fmcs1/readings/matl... Basic probability theory - http://homepages.inf.ed.ac.uk/sgwater/teaching/general/probability.pdf http://homepages.inf.ed.ac.uk/sgwater/teaching/general/proba... Mathematics for Machine Learning - https://pdfs.semanticscholar.org/910e/3118b50f426e5e840561e1f9f1fdd67e5724.pdf https://pdfs.semanticscholar.org/910e/3118b50f426e5e840561e1... Math for Machine Learning - https://www.umiacs.umd.edu/~hal/courses/2013S_ML/math4ml.pdf https://www.umiacs.umd.edu/~hal/courses/2013S_ML/math4ml.pdf
- mayankkaizen 9y ago
  Yes I have read his comment and added a reply there. To reiterate my point, I asked for crash courses not because I wanted to make quick progress but because I wanted to have an idea of what kind of mathematics is needed. In a way I was looking for some syllabus with some description added. Thank you for posting these links. I'll definitely go through these.
usgroup 9y ago
Undergrad lin algebra and calculus ... start there! Although there is no skimmable view of those worth a damn. You need 2-3 months @ 8 hours a day. Stats thereafter is pie, ML low hanging fruit will come easily. Good inference and learning from data on the other hand is experience: expect years, no shortcuts. In fact, suffer. It’ll make the journey as informative as possible.
zengid 9y ago
This article on matrix calculus seems pretty good for someone familiar with ML and DL concepts but not on the maths: http://parrt.cs.usfca.edu/doc/matrix-calculus/index.html#sec3 http://parrt.cs.usfca.edu/doc/matrix-calculus/index.html#sec...
sushanthiray 9y ago
Take a look at the Coursera specialization: Mathematics for Machine Learning [1]. The specialization isn't free but you can certainly apply for financial aid. [1]: https://www.coursera.org/specializations/mathematics-machine-learning https://www.coursera.org/specializations/mathematics-machine...
- astrodev 9y ago
  Yes! It's a new course that will be open for enrolment soon. I think it's exactly what most people here are looking for. There is the financial aid and normally there is the option to watch the lectures and see the assignment for free. On the other hand, the cost of the courses rarely exceeds $50.
uptownfunk 9y ago
the strang lectures on LA, any decent calculus book (khan academy or something maybe too) and then i would jump into ISLR and the karpathy Stanford lectures on DL, the rest you learn as you go adhoc.
- usgroup 9y ago
  One can't over emphasise: LA and Calc. Do that! The rest will be a cake walk. But you need LA from matrices to spectral decomposition and Calc from differentiation to Laplace transforms; so no easy feat. You'll need a fair amount of "give a damn" to put yourself through it.
imakecomments 9y ago
Any PreCalculus text -> Stewart Calculus -> Strang + Axler Linear Algebra -> a Calculus based Statistics & Probability book. Do the problems in the books. Read each chapter. If you do that you'll know Calculus and Linear Algebra better than most. If you want to then move up from there study Real Analysis and higher level math like Measure theory (but this may not be necessary for your purposes).
gtani 9y ago
Similar quesion, aimed at math majors: https://www.reddit.com/r/math/comments/81px9v/what_are_some_good_mathematical_books_should_i/ https://www.reddit.com/r/math/comments/81px9v/what_are_some_... Really, if you're not going to go to e.g. community college where you can do a lot of this (LA, prob/stats, calc) your best shot is to find study groups in Data science meetups, etc which are plentiful in major cities since many people have the same impulse. Truthfully, self study rarely gets far if people never went beyond Calc 2. Here's a blog about one guy's adventure, read the part where he talks about Shores' LA text(not a bad book by the way but yes, lots of typos): https://news.ycombinator.com/item?id=8996024 https://news.ycombinator.com/item?id=8996024
mindcrash 9y ago
This got trending on Github today. It might help: https://github.com/llSourcell/Learn_Machine_Learning_in_3_Months https://github.com/llSourcell/Learn_Machine_Learning_in_3_Mo... First month covers the basics (calculus, algebra, probability, algorithms) Second month is focused around coding (Data science in Python) and putting the things learned in the first month in a ML context (Siraj's Math of Intelligence course on YouTube) Third month is all around Deep Learning.
ivan_ah 9y ago
Check out my books on CALCULUS+MECHANICS, and LINEAR ALGEBRA: https://minireference.com/ https://minireference.com/ They are not free, but not expensive. It's like 2 years worth of undergraduate maths packed into two small books. Previews are free though, and might be useful if all you need is an overview: https://minireference.com/static/excerpts/noBSguide_v5_preview.pdf https://minireference.com/static/excerpts/noBSguide_v5_previ... https://minireference.com/static/excerpts/noBSguide2LA_preview.pdf https://minireference.com/static/excerpts/noBSguide2LA_previ... This is also good: http://ml-cheatsheet.readthedocs.io/en/latest/ http://ml-cheatsheet.readthedocs.io/en/latest/
mithunmanohar1 9y ago
Week 1 Linear Algebra https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/ https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra... Week 2 Calculus https://www.youtube.com/playlist?list=PLZHQObOWTQDMsr9K-rj53DwVRMYO3t5Yr https://www.youtube.com/playlist?list=PLZHQObOWTQDMsr9K-rj53... Week 3 Probability https://www.edx.org/course/introduction-probability-science-mitx-6-041x-2 https://www.edx.org/course/introduction-probability-science-... Week 4 Algorithms https://www.coursera.org/courses?languages=en&query=Algorithm%20design%20and%20analysis https://www.coursera.org/courses?languages=en&query=Algorith...

103 points by mayankkaizen 9y ago

bjourne 9y ago

Read these books! * http://www-bcf.usc.edu/~gareth/ISL/ISLR%20First%20Printing.pdf http://www-bcf.usc.edu/~gareth/ISL/ISLR%20First%20Printing.p... * http://web4.cs.ucl.ac.uk/staff/s.prince/book/book.pdf http://web4.cs.ucl.ac.uk/staff/s.prince/book/book.pdf * http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%... I am not an expert in ML. I have heard experts in ML call the above books introductory.

abhinavkits501 9y ago

https://developers.google.com/machine-learning/crash-course/ml-intro https://developers.google.com/machine-learning/crash-course/...

BiancaDelRio 9y ago

http://www.cs.cornell.edu/jeh/book.pdf http://www.cs.cornell.edu/jeh/book.pdf

mayankkaizen 9y ago
Thanks!
WalterGR 9y ago
Foundations of Data Science Avrim Blum, John Hopcroft, and Ravindran Kannan Copyright 2015

dsacco 9y ago

I’m sorry to say this because I observe that we get these questions somewhat frequently, but I don’t think what you’re asking for - in the format, goals and speed you’d like it - is a thing. That’s a pretty discouraging statement, so let me try to be more helpful. The mathematics comprising contemporary machine learning and data analysis primarily consists of linear algebra, calculus, mathematical statistics and, at the high end, probability theory. There are boutique efforts to use things like topology but that’s non-standard. As it stands, your question is underspecified, which is what I repeat for most of these questions. The answers you receive here are going to variously interpret what you’re actually looking for and prescribe based on the author’s interpretation and intuition. It would be more useful if you explained exactly what your goals are. Here’s precisely the problem: you apparently don’t know these mathematics already, but you’re asking for something that “doesn’t necessarily go into depth.” How much depth is too much, and how would you know that exactly given your present unknown unknowns? If you explain what you actually want to achieve, we might be able to 1. Tell you that you don’t actually need that “top level view” to achieve what you’d like, 2. Tell you that such a top level view is not nearly enough for what you’d like to do, or 3. Tell you that a top level view is coherent, and optimize the best materials for you to learn from based on what you want to do. It would also help us make recommendations if you explained what your current level of mathematical background looks like. Have you taken linear algebra at least once? Exactly how basic do our resources have to be? Different textbooks written for different audiences can variously explain the same concept in two pages or 10 pages, and they can emphasize different things. I’d like to help, and I probably can, but it would go a long way if you could tell us what your end goal is instead of what you interpret as the next step towards that goal. Then we can provide resources based on your mathematical maturity.

jxub 9y ago
Topology for ML sounds amazing. Got any papers/links?
- dsacco 9y ago
  Sure, try this: http://math.ucr.edu/home/baez/information/ http://math.ucr.edu/home/baez/information/. In a nutshell, you take probability distributions and project them onto topological manifolds, such that the distribution consists of points on the manifold. You can find more by searching for key words like "information geometry" or "differential geometry machine learning".
  - mlevental 9y ago
    there's also the stuff that ayasdi and carlsson were/are doing but i never really saw the point of that (e.g. just compute connected components instead persistent homology).
mayankkaizen 9y ago
OP here. Thanks for taking time to reply. Really thank you. Obviously I know very little about these areas and that is why I posted such a question. By posing this question, I didn't mean that I wanted to 'get there' quickly. In fact I am in no hurry and willing to go as deep as possible. What I actually mean to ask is that I was looking for a kind of syllabus/crash courses so that I can get an idea about what kind of mathematics is involved in these areas. Based on that, I'll be able to start my studies in much more organized way. Also, from there on, I myself be able to figure out how to look for further resources. This is the style of learning I usually follow. Hope you understand my point.

sdiq 9y ago

First, please read and internalize dsacco's comment. In the meantime, the following might help. But from experience, if your math is already shaky, you would certainly need to look for resources covering everything listed in a more in depth manner. Basic calculus refresher - http://www.stat.wisc.edu/~ifischer/calculus.pdf http://www.stat.wisc.edu/~ifischer/calculus.pdf Learning from Data - http://www.inf.ed.ac.uk/teaching/courses/fmcs1/readings/matlab-barber.pdf http://www.inf.ed.ac.uk/teaching/courses/fmcs1/readings/matl... Basic probability theory - http://homepages.inf.ed.ac.uk/sgwater/teaching/general/probability.pdf http://homepages.inf.ed.ac.uk/sgwater/teaching/general/proba... Mathematics for Machine Learning - https://pdfs.semanticscholar.org/910e/3118b50f426e5e840561e1f9f1fdd67e5724.pdf https://pdfs.semanticscholar.org/910e/3118b50f426e5e840561e1... Math for Machine Learning - https://www.umiacs.umd.edu/~hal/courses/2013S_ML/math4ml.pdf https://www.umiacs.umd.edu/~hal/courses/2013S_ML/math4ml.pdf

mayankkaizen 9y ago
Yes I have read his comment and added a reply there. To reiterate my point, I asked for crash courses not because I wanted to make quick progress but because I wanted to have an idea of what kind of mathematics is needed. In a way I was looking for some syllabus with some description added. Thank you for posting these links. I'll definitely go through these.

usgroup 9y ago

Undergrad lin algebra and calculus ... start there! Although there is no skimmable view of those worth a damn. You need 2-3 months @ 8 hours a day. Stats thereafter is pie, ML low hanging fruit will come easily. Good inference and learning from data on the other hand is experience: expect years, no shortcuts. In fact, suffer. It’ll make the journey as informative as possible.

zengid 9y ago

This article on matrix calculus seems pretty good for someone familiar with ML and DL concepts but not on the maths: http://parrt.cs.usfca.edu/doc/matrix-calculus/index.html#sec3 http://parrt.cs.usfca.edu/doc/matrix-calculus/index.html#sec...

sushanthiray 9y ago

Take a look at the Coursera specialization: Mathematics for Machine Learning [1]. The specialization isn't free but you can certainly apply for financial aid. [1]: https://www.coursera.org/specializations/mathematics-machine-learning https://www.coursera.org/specializations/mathematics-machine...

astrodev 9y ago
Yes! It's a new course that will be open for enrolment soon. I think it's exactly what most people here are looking for. There is the financial aid and normally there is the option to watch the lectures and see the assignment for free. On the other hand, the cost of the courses rarely exceeds $50.

uptownfunk 9y ago

the strang lectures on LA, any decent calculus book (khan academy or something maybe too) and then i would jump into ISLR and the karpathy Stanford lectures on DL, the rest you learn as you go adhoc.

usgroup 9y ago
One can't over emphasise: LA and Calc. Do that! The rest will be a cake walk. But you need LA from matrices to spectral decomposition and Calc from differentiation to Laplace transforms; so no easy feat. You'll need a fair amount of "give a damn" to put yourself through it.

imakecomments 9y ago

Any PreCalculus text -> Stewart Calculus -> Strang + Axler Linear Algebra -> a Calculus based Statistics & Probability book. Do the problems in the books. Read each chapter. If you do that you'll know Calculus and Linear Algebra better than most. If you want to then move up from there study Real Analysis and higher level math like Measure theory (but this may not be necessary for your purposes).

gtani 9y ago

Similar quesion, aimed at math majors: https://www.reddit.com/r/math/comments/81px9v/what_are_some_good_mathematical_books_should_i/ https://www.reddit.com/r/math/comments/81px9v/what_are_some_... Really, if you're not going to go to e.g. community college where you can do a lot of this (LA, prob/stats, calc) your best shot is to find study groups in Data science meetups, etc which are plentiful in major cities since many people have the same impulse. Truthfully, self study rarely gets far if people never went beyond Calc 2. Here's a blog about one guy's adventure, read the part where he talks about Shores' LA text(not a bad book by the way but yes, lots of typos): https://news.ycombinator.com/item?id=8996024 https://news.ycombinator.com/item?id=8996024

mindcrash 9y ago

This got trending on Github today. It might help: https://github.com/llSourcell/Learn_Machine_Learning_in_3_Months https://github.com/llSourcell/Learn_Machine_Learning_in_3_Mo... First month covers the basics (calculus, algebra, probability, algorithms) Second month is focused around coding (Data science in Python) and putting the things learned in the first month in a ML context (Siraj's Math of Intelligence course on YouTube) Third month is all around Deep Learning.

ivan_ah 9y ago

Check out my books on CALCULUS+MECHANICS, and LINEAR ALGEBRA: https://minireference.com/ https://minireference.com/ They are not free, but not expensive. It's like 2 years worth of undergraduate maths packed into two small books. Previews are free though, and might be useful if all you need is an overview: https://minireference.com/static/excerpts/noBSguide_v5_preview.pdf https://minireference.com/static/excerpts/noBSguide_v5_previ... https://minireference.com/static/excerpts/noBSguide2LA_preview.pdf https://minireference.com/static/excerpts/noBSguide2LA_previ... This is also good: http://ml-cheatsheet.readthedocs.io/en/latest/ http://ml-cheatsheet.readthedocs.io/en/latest/

mithunmanohar1 9y ago

Week 1 Linear Algebra https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/ https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra... Week 2 Calculus https://www.youtube.com/playlist?list=PLZHQObOWTQDMsr9K-rj53DwVRMYO3t5Yr https://www.youtube.com/playlist?list=PLZHQObOWTQDMsr9K-rj53... Week 3 Probability https://www.edx.org/course/introduction-probability-science-mitx-6-041x-2 https://www.edx.org/course/introduction-probability-science-... Week 4 Algorithms https://www.coursera.org/courses?languages=en&query=Algorithm%20design%20and%20analysis https://www.coursera.org/courses?languages=en&query=Algorith...