7 ms·

I'm e-learning Linear Algebra right now to have a good math foundation for Machine Learning. I was a History and Sociology major in college - so I didn't take

I'm e-learning Linear Algebra right now to have a good math foundation for Machine Learning. I was a History and Sociology major in college - so I didn't take any math. If you are like me, and working off an initial base of high school math, I would recommend the following (all free): Linear Algebra Foundations to Frontiers (UT Austin) Course: https://www.edx.org/course/linear-algebra-foundations-to-frontiers https://www.edx.org/course/linear-algebra-foundations-to-fro... Comments: This was a great starting place for me. Good interactive HW exercises, very clear instruction and time-efficient. Linear Algebra (MIT OpenCourseware) Course: https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/ https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra... Comments: This course is apparently the holy grail course for Intro Linear Algebra. One of my colleagues, who did an MS in EE at MIT, said Gilbert Strang was the best teacher he had. I started off with this but had to rewind to the UT class because I didn't have some of the fundamentals (e.g. how to calc a dot product). I'm personally 15% through this, but enjoying it. Linear Algebra Review PDF (Stanford CS229) Link: http://cs229.stanford.edu/section/cs229-linalg.pdf http://cs229.stanford.edu/section/cs229-linalg.pdf Comments: This is the set of Linear Algebra review materials they go over at the beginning of Stanford's machine learning class (CS229). This is my workback to know I'm tracking to the right set of knowledge, and thus far, the courses have done a great job of doing so.

flor1s 8y ago
Don't forget to review calculus as well. Khan Academy is a good start for learning about single variable calculus (http://www.khanacademy.org http://www.khanacademy.org), but their content on multivariable calculus is a bit lacking (neural networks / deep learning use the concept of the derivatives and the gradient a lot). A good supplement for multivariable calculus would be Terence Parr and Jeremy Howard's article on "All the matrix calculus you need for deep learning": https://explained.ai/matrix-calculus/index.html https://explained.ai/matrix-calculus/index.html
- muhneesh 8y ago
  Thanks - I am doing that as well! I've been using MIT OpenCourseware for single variable calculus (and will do the same for multivariable). I fenced the parent post to Linear Algebra to not go too far away from the OP. I will certainly check out the Terrence Parr / Jeremy Howard site, and am super familiar with Khan Academy.
  - anonymous_i 8y ago
    I came across-Calculus Made Easy by Silvanus Thompson,on someones twitter feed. Published in 1910 and far less scary and far more interesting to read than a lot of math text books. https://www.gutenberg.org/files/33283/33283-pdf.pdf https://www.gutenberg.org/files/33283/33283-pdf.pdf
    - FranzFerdiNaN 8y ago
      This should be your first calculus book if you are learning from scratch. Much better than those 1000+ page behemoths colleges use.
    - claudiawerner 8y ago
      Similarly, Stroud's "Engineering Mathematics" takes you right from addition all the way up to Fourier transforms... a fantatsic book.
    - ajot 8y ago
      There is also a web goodlooking version, discussed previously in HN: https://news.ycombinator.com/item?id=18250034 https://news.ycombinator.com/item?id=18250034
    - Cheyana 8y ago
      “Considering how many fools can calculate, it is surprising that it should be thought either a difficult or a tedious task for any other fool to learn how to master the same tricks.” I really wish technical books were still written like this. Though if Thompson posted this on HN as a comment he probably would have been downvoted.
  - 8y ago
earthicus 8y ago
> This course [Strang] is apparently the holy grail course for Intro Linear Algebra. I haven't watched his lectures, but I TA'd a linear algebra course that used his text book, and strongly disliked his presentation. I've heard that's a fairly common reaction actually - it's one of those love it or hate it books. I'm bringing it up because if you (or someone else reading this) turn out to be in the group that doesn't love it, you should not give up on loving linear algebra! You are definitely still allowed to have a different 'holy grail course'!
- muhneesh 8y ago
  Awesome input! Learning isn't linear (tee-hee...)
- the_clarence 8y ago
  Gilbert Strang is probably the best teacher on videos, up there with Dan Boneh.
- selimthegrim 8y ago
  Where’s the love for Lax?
  - earthicus 8y ago
    Page after page of mathematical insights and delights! I've never had the opportunity to work through it systematically, but have frequently read excerpts and have never been let down. I would expect nothing less from a figure so great as Lax! It's worth pointing out in the context of this discussion that the book is, by the author's own design, not an introduction to linear algebra. It is a second course that Lax used to teach his advanced undergraduates and beginning graduate students at the Courant Institute. For example, OP with a high school math background will surely be very puzzled by page two, when a linear space is defined as a field 'acting on' a group. Which is, i think, the 'right' way of thinking about the algebraic structure, in the sense that it greatly simplifies all the intricate moving parts of linear algebra. Anyhow, I second your recommendation!
peteretep 8y ago
A bit weird to add a negative review, but here goes: https://www.coursera.org/learn/linear-algebra-machine-learning https://www.coursera.org/learn/linear-algebra-machine-learni... Is _not_ a good introduction. The instructors are all over the damn place, and you will spend much of your time finding better explanations from other sources. Wish I hadn't started with this. On the plus side, you will get a certificate at the end.
- pumanoir 8y ago
  Indeed weird, because the course you mentioned is actually excellent. However, it was designed for people who had (somehow) already seen the subjects in an abstract and unapplied setting (such as a math class at uni). They refresh or refocus the subjects with a geometric intuition and with some concrete applications in mind; which I found quite useful and beautiful. This class is more like a more developed version of 3b1b videos on LA.
  - elbear 8y ago
    It's funny, because as I was reading your comment, I was thinking of 3b1b. He's doing great work by visualizing abstract concepts, but I think what he does mainly helps people who have already gone through the material. If it's your first time encountering the topic, you'll likely feel lost or not see the point. What 3b1b does still brings a lot of value, so I don't want to take away anything from his work.
kharak 8y ago
Out of curiosity, do you feel you can compete with people who have advanced degrees in more quantitative sciences? Although I'm in the process of plugging several holes in my own math education, I don't believe I'd be able to get any interesting, ML related jobs. I also don't see myself able to perform well in comparison, given that I lack the mathematical intuition one builds after several years of (almost) daily practice. (I hope I don't sound discouraging, relearning math has been quit fun so far and made me able to understand more of everything).
- muhneesh 8y ago
  The direct answer is - not at the research level but yes, in terms of application as the field matures and abstracts over time. There are also adjacent jobs to ML engineer (product management and so on...).
hnuser355 8y ago
If you care about anything that runs on a computer, linear algebra is one of the best maths.
roymurdock 8y ago
Would be interested to hear why you’re studying machine learning. Do you see important problems you think it can solve, are you looking to make more $$ as an ML data scientist, or just generally interested in stats/data?
- muhneesh 8y ago
  Part of a broader effort - I committed to learning to code about 3 years ago. At that point in time, I didn't really know why I was doing it... really out of curiosity. I kept going because it was addicting, and really an antithesis to my day job at the time (investment banking) - which I felt was corporate / bureaucratic and unintellectual. That said, eventually, I want to start a startup. I'm building out small side projects now. I'm generally comfy with web + mobile dev, and I wanted to upskill in a "newer" technology that was more "mathy".
  - all2 8y ago
    I bet you could flow-chart your entire firm into a process that's mostly automated. Something like wealthfront, maybe?
  - roymurdock 8y ago
    Thanks for sharing, good luck on your journey
rasmus1610 8y ago
I just love the MIT linear algebra course with Gilbert Strang. Awesome teacher
dsiegel2275 8y ago
I agree, Strang's Linear Algebra course is excellent. I worked through the entire course in March/April this year. I just completed my final exam at CMU in their graduate intro to ML class (10-601). Having gone through the LA course was essential to my success. But equally important (if not more) to ML is a solid foundation in probability.
jackallis 8y ago
did you find LAFF too mathy? i got turned off by the mathyness of it and i quit in 2 weeks. does it get any better? All the math notations and lines got so dry that i vapourised trying to understand.
- muhneesh 8y ago
  I didn't think about it during the time. It's a fair comment, and probably true. What it did really well (for me) was integrate HW with each lecture video, and start at a really basic foundation. It took me from 0 -> something.
ivan_ah 8y ago
That's a pretty good list, here are some things I'd add. Amazing js visualizations/manipulatives for many LA concepts: http://immersivemath.com/ila/index.html http://immersivemath.com/ila/index.html LA Concept map: https://minireference.com/static/tutorials/conceptmap.pdf#page=2 https://minireference.com/static/tutorials/conceptmap.pdf#pa... (so you'll know what there is to learn) Condensed 4-page tutorial: https://minireference.com/static/tutorials/linear_algebra_in_4_pages.pdf https://minireference.com/static/tutorials/linear_algebra_in... (in case you're short on time) And here is an excerpt from my book: https://minireference.com/static/excerpts/noBSguide2LA_preview.pdf https://minireference.com/static/excerpts/noBSguide2LA_previ... (won't post a link to it here, but check on amazon if interested)
- muhneesh 8y ago
  Wow - nice stuff. I'm a fan of the reference map.
lalaithion 8y ago
3blue1brown has a great youtube series on both Calculus and Linear Algebra that provide excellent intuitive backing for the concepts in both areas.
espeed 8y ago
Good recommendations. In addition to the UT, MIT and Stanford courses you recommend above, for developing your visual intuition, 3Blue1Brown's Essence of Linear Algebra video series is second to none. [0] Another good one is MathTheBeautiful [1] by MIT alum Pavel Grinfeld [2]. He approaches Linear Algebra from a geometric perspective as well, but with more emphasis on the mechanics of solving equations. He has a ton of videos organized into several courses, ranging from in-depth Intro to Linear Algebra courses to more advanced courses on PDEs and Tensor Calculus. Esp note his video on Legendre polynomials [3] and Why {1,x,x²} Is a Terrible Basis: https://www.youtube.com/watch?v=pYoGYQOXqTk&index=14&list=PL... https://www.youtube.com/watch?v=pYoGYQOXqTk&index=14&list=PL.... Gilbert Strang was Greenfield's PhD advisor: https://dspace.mit.edu/handle/1721.1/29345 https://dspace.mit.edu/handle/1721.1/29345. Pavel has a clear and precise teaching style like Strang, and he makes reference to Prof's Strang and his MIT course from time to time. NB: Prof Strang has a new book Linear Algebra and Learning from Data that just went to press and will be available in print by mid Jan 2019. A few chapters are available online now, and the video lectures from the new MIT course should on YouTube in a few weeks. [4] [0] Essence of Linear Video Series (3Blue1Brown): https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw/playlists https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw/pla... [1] MathTheBeautiful https://www.youtube.com/watch?v=pYoGYQOXqTk&index=14&list=PL https://www.youtube.com/watch?v=pYoGYQOXqTk&index=14&list=PL [2] https://en.wikipedia.org/wiki/Pavel_Grinfeld https://en.wikipedia.org/wiki/Pavel_Grinfeld [3] https://en.wikipedia.org/wiki/Legendre_polynomials https://en.wikipedia.org/wiki/Legendre_polynomials [4] MIT Linear Algebra and Learning from Data (2018) http://math.mit.edu/~gs/learningfromdata/ http://math.mit.edu/~gs/learningfromdata/
thanatropism 8y ago
What's the difference between learning and e-learning? Is the latter faster?
- CBLT 8y ago
  Orders of magnitude cheaper.

flor1s 8y ago

Don't forget to review calculus as well. Khan Academy is a good start for learning about single variable calculus (http://www.khanacademy.org http://www.khanacademy.org), but their content on multivariable calculus is a bit lacking (neural networks / deep learning use the concept of the derivatives and the gradient a lot). A good supplement for multivariable calculus would be Terence Parr and Jeremy Howard's article on "All the matrix calculus you need for deep learning": https://explained.ai/matrix-calculus/index.html https://explained.ai/matrix-calculus/index.html

muhneesh 8y ago
Thanks - I am doing that as well! I've been using MIT OpenCourseware for single variable calculus (and will do the same for multivariable). I fenced the parent post to Linear Algebra to not go too far away from the OP. I will certainly check out the Terrence Parr / Jeremy Howard site, and am super familiar with Khan Academy.
- anonymous_i 8y ago
  I came across-Calculus Made Easy by Silvanus Thompson,on someones twitter feed. Published in 1910 and far less scary and far more interesting to read than a lot of math text books. https://www.gutenberg.org/files/33283/33283-pdf.pdf https://www.gutenberg.org/files/33283/33283-pdf.pdf
  - FranzFerdiNaN 8y ago
    This should be your first calculus book if you are learning from scratch. Much better than those 1000+ page behemoths colleges use.
  - claudiawerner 8y ago
    Similarly, Stroud's "Engineering Mathematics" takes you right from addition all the way up to Fourier transforms... a fantatsic book.
  - ajot 8y ago
    There is also a web goodlooking version, discussed previously in HN: https://news.ycombinator.com/item?id=18250034 https://news.ycombinator.com/item?id=18250034
  - Cheyana 8y ago
    “Considering how many fools can calculate, it is surprising that it should be thought either a difficult or a tedious task for any other fool to learn how to master the same tricks.” I really wish technical books were still written like this. Though if Thompson posted this on HN as a comment he probably would have been downvoted.
- 8y ago

earthicus 8y ago

> This course [Strang] is apparently the holy grail course for Intro Linear Algebra. I haven't watched his lectures, but I TA'd a linear algebra course that used his text book, and strongly disliked his presentation. I've heard that's a fairly common reaction actually - it's one of those love it or hate it books. I'm bringing it up because if you (or someone else reading this) turn out to be in the group that doesn't love it, you should not give up on loving linear algebra! You are definitely still allowed to have a different 'holy grail course'!

muhneesh 8y ago
Awesome input! Learning isn't linear (tee-hee...)
the_clarence 8y ago
Gilbert Strang is probably the best teacher on videos, up there with Dan Boneh.
selimthegrim 8y ago
Where’s the love for Lax?
- earthicus 8y ago
  Page after page of mathematical insights and delights! I've never had the opportunity to work through it systematically, but have frequently read excerpts and have never been let down. I would expect nothing less from a figure so great as Lax! It's worth pointing out in the context of this discussion that the book is, by the author's own design, not an introduction to linear algebra. It is a second course that Lax used to teach his advanced undergraduates and beginning graduate students at the Courant Institute. For example, OP with a high school math background will surely be very puzzled by page two, when a linear space is defined as a field 'acting on' a group. Which is, i think, the 'right' way of thinking about the algebraic structure, in the sense that it greatly simplifies all the intricate moving parts of linear algebra. Anyhow, I second your recommendation!

peteretep 8y ago

A bit weird to add a negative review, but here goes: https://www.coursera.org/learn/linear-algebra-machine-learning https://www.coursera.org/learn/linear-algebra-machine-learni... Is _not_ a good introduction. The instructors are all over the damn place, and you will spend much of your time finding better explanations from other sources. Wish I hadn't started with this. On the plus side, you will get a certificate at the end.

pumanoir 8y ago
Indeed weird, because the course you mentioned is actually excellent. However, it was designed for people who had (somehow) already seen the subjects in an abstract and unapplied setting (such as a math class at uni). They refresh or refocus the subjects with a geometric intuition and with some concrete applications in mind; which I found quite useful and beautiful. This class is more like a more developed version of 3b1b videos on LA.
- elbear 8y ago
  It's funny, because as I was reading your comment, I was thinking of 3b1b. He's doing great work by visualizing abstract concepts, but I think what he does mainly helps people who have already gone through the material. If it's your first time encountering the topic, you'll likely feel lost or not see the point. What 3b1b does still brings a lot of value, so I don't want to take away anything from his work.

kharak 8y ago

Out of curiosity, do you feel you can compete with people who have advanced degrees in more quantitative sciences? Although I'm in the process of plugging several holes in my own math education, I don't believe I'd be able to get any interesting, ML related jobs. I also don't see myself able to perform well in comparison, given that I lack the mathematical intuition one builds after several years of (almost) daily practice. (I hope I don't sound discouraging, relearning math has been quit fun so far and made me able to understand more of everything).

muhneesh 8y ago
The direct answer is - not at the research level but yes, in terms of application as the field matures and abstracts over time. There are also adjacent jobs to ML engineer (product management and so on...).

hnuser355 8y ago

If you care about anything that runs on a computer, linear algebra is one of the best maths.

roymurdock 8y ago

Would be interested to hear why you’re studying machine learning. Do you see important problems you think it can solve, are you looking to make more $$ as an ML data scientist, or just generally interested in stats/data?

muhneesh 8y ago
Part of a broader effort - I committed to learning to code about 3 years ago. At that point in time, I didn't really know why I was doing it... really out of curiosity. I kept going because it was addicting, and really an antithesis to my day job at the time (investment banking) - which I felt was corporate / bureaucratic and unintellectual. That said, eventually, I want to start a startup. I'm building out small side projects now. I'm generally comfy with web + mobile dev, and I wanted to upskill in a "newer" technology that was more "mathy".
- all2 8y ago
  I bet you could flow-chart your entire firm into a process that's mostly automated. Something like wealthfront, maybe?
- roymurdock 8y ago
  Thanks for sharing, good luck on your journey

rasmus1610 8y ago

I just love the MIT linear algebra course with Gilbert Strang. Awesome teacher

dsiegel2275 8y ago

I agree, Strang's Linear Algebra course is excellent. I worked through the entire course in March/April this year. I just completed my final exam at CMU in their graduate intro to ML class (10-601). Having gone through the LA course was essential to my success. But equally important (if not more) to ML is a solid foundation in probability.

jackallis 8y ago

did you find LAFF too mathy? i got turned off by the mathyness of it and i quit in 2 weeks. does it get any better? All the math notations and lines got so dry that i vapourised trying to understand.

muhneesh 8y ago
I didn't think about it during the time. It's a fair comment, and probably true. What it did really well (for me) was integrate HW with each lecture video, and start at a really basic foundation. It took me from 0 -> something.

ivan_ah 8y ago

That's a pretty good list, here are some things I'd add. Amazing js visualizations/manipulatives for many LA concepts: http://immersivemath.com/ila/index.html http://immersivemath.com/ila/index.html LA Concept map: https://minireference.com/static/tutorials/conceptmap.pdf#page=2 https://minireference.com/static/tutorials/conceptmap.pdf#pa... (so you'll know what there is to learn) Condensed 4-page tutorial: https://minireference.com/static/tutorials/linear_algebra_in_4_pages.pdf https://minireference.com/static/tutorials/linear_algebra_in... (in case you're short on time) And here is an excerpt from my book: https://minireference.com/static/excerpts/noBSguide2LA_preview.pdf https://minireference.com/static/excerpts/noBSguide2LA_previ... (won't post a link to it here, but check on amazon if interested)

muhneesh 8y ago
Wow - nice stuff. I'm a fan of the reference map.

lalaithion 8y ago

3blue1brown has a great youtube series on both Calculus and Linear Algebra that provide excellent intuitive backing for the concepts in both areas.

espeed 8y ago

Good recommendations. In addition to the UT, MIT and Stanford courses you recommend above, for developing your visual intuition, 3Blue1Brown's Essence of Linear Algebra video series is second to none. [0] Another good one is MathTheBeautiful [1] by MIT alum Pavel Grinfeld [2]. He approaches Linear Algebra from a geometric perspective as well, but with more emphasis on the mechanics of solving equations. He has a ton of videos organized into several courses, ranging from in-depth Intro to Linear Algebra courses to more advanced courses on PDEs and Tensor Calculus. Esp note his video on Legendre polynomials [3] and Why {1,x,x²} Is a Terrible Basis: https://www.youtube.com/watch?v=pYoGYQOXqTk&index=14&list=PL... https://www.youtube.com/watch?v=pYoGYQOXqTk&index=14&list=PL.... Gilbert Strang was Greenfield's PhD advisor: https://dspace.mit.edu/handle/1721.1/29345 https://dspace.mit.edu/handle/1721.1/29345. Pavel has a clear and precise teaching style like Strang, and he makes reference to Prof's Strang and his MIT course from time to time. NB: Prof Strang has a new book Linear Algebra and Learning from Data that just went to press and will be available in print by mid Jan 2019. A few chapters are available online now, and the video lectures from the new MIT course should on YouTube in a few weeks. [4] [0] Essence of Linear Video Series (3Blue1Brown): https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw/playlists https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw/pla... [1] MathTheBeautiful https://www.youtube.com/watch?v=pYoGYQOXqTk&index=14&list=PL https://www.youtube.com/watch?v=pYoGYQOXqTk&index=14&list=PL [2] https://en.wikipedia.org/wiki/Pavel_Grinfeld https://en.wikipedia.org/wiki/Pavel_Grinfeld [3] https://en.wikipedia.org/wiki/Legendre_polynomials https://en.wikipedia.org/wiki/Legendre_polynomials [4] MIT Linear Algebra and Learning from Data (2018) http://math.mit.edu/~gs/learningfromdata/ http://math.mit.edu/~gs/learningfromdata/

thanatropism 8y ago

What's the difference between learning and e-learning? Is the latter faster?

CBLT 8y ago
Orders of magnitude cheaper.