4 ms·

Linear Algebra for Deep Learning: Matrix Algebra

131 points by shogunmike 9y ago

adamkochanowicz 9y ago
Awesome! I have some Khan Academy courses queued up to learn linear algebra, but this is more targeted and useful.
- paulgb 9y ago
  I'd highly recommend this set of videos: https://youtu.be/kjBOesZCoqc https://youtu.be/kjBOesZCoqc
  - brudgers 9y ago
    Also Gilbert Strang's lectures: https://www.youtube.com/watch?v=ZK3O402wf1c&list=PL49CF3715CB9EF31D https://www.youtube.com/watch?v=ZK3O402wf1c&list=PL49CF3715C...
- nerdponx 9y ago
  The best intro to linear algebra, bar none, is the MIT OpenCourseware Intro to Linear Algebra, recordings of a course taught in 2005 by Prof. Gilbert Strang: https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/index.htm https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra...
  - shogunmike 9y ago
    Indeed - I recommended Strang's book in this article and the previous one.
    - xrange 9y ago
      The Strang book: "Introduction to Linear Algebra" does seem to have quite a few "high quality" 1-star reviews. They mostly seem to cluster around this book not being good for a linear algebra introduction, for someone new to the field. Typical comments: "It seems the reviewers who think this book is wonderful for non-math majors are math majors!" https://www.amazon.com/Introduction-Linear-Algebra-Gilbert-Strang/product-reviews/0980232775/ref=cm_cr_getr_d_paging_btm_next_2?ie=UTF8&filterByStar=one_star&reviewerType=all_reviews&pageNumber=2#reviews-filter-bar https://www.amazon.com/Introduction-Linear-Algebra-Gilbert-S... ...I of course expect that any popular book will have 1 star reviews, especially by people who had a hard time with the material. Anyone have thoughts whether the Strang book is better as a supplement to other texts in coming up to speed on Linear Algebra? Other book recommendations?
deleted 9y ago
[deleted]
nicolewhite 9y ago
This is a nice intro to linear algebra but doesn't really get into how it's related to deep learning. A good read is Vector, Matrix, and Tensor Derivatives[0], which goes into the linear algebra needed in multi-layer neural networks, specifically during the backpropogation phase. [0] http://cs231n.stanford.edu/vecDerivs.pdf http://cs231n.stanford.edu/vecDerivs.pdf
- shogunmike 9y ago
  You're correct - it doesn't get into the deep learning aspect yet. This article is in fact the second part of a larger planned series. The idea is to present more depth than a "quick refresher" that seems to be common to a lot of blog posts, but far less material than would be found in a 10-week (or single semester) undergraduate course. Thanks for the Stanford link. I'll check it out.

131 points by shogunmike 9y ago

adamkochanowicz 9y ago

Awesome! I have some Khan Academy courses queued up to learn linear algebra, but this is more targeted and useful.

paulgb 9y ago
I'd highly recommend this set of videos: https://youtu.be/kjBOesZCoqc https://youtu.be/kjBOesZCoqc
- brudgers 9y ago
  Also Gilbert Strang's lectures: https://www.youtube.com/watch?v=ZK3O402wf1c&list=PL49CF3715CB9EF31D https://www.youtube.com/watch?v=ZK3O402wf1c&list=PL49CF3715C...
nerdponx 9y ago
The best intro to linear algebra, bar none, is the MIT OpenCourseware Intro to Linear Algebra, recordings of a course taught in 2005 by Prof. Gilbert Strang: https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/index.htm https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra...
- shogunmike 9y ago
  Indeed - I recommended Strang's book in this article and the previous one.
  - xrange 9y ago
    The Strang book: "Introduction to Linear Algebra" does seem to have quite a few "high quality" 1-star reviews. They mostly seem to cluster around this book not being good for a linear algebra introduction, for someone new to the field. Typical comments: "It seems the reviewers who think this book is wonderful for non-math majors are math majors!" https://www.amazon.com/Introduction-Linear-Algebra-Gilbert-Strang/product-reviews/0980232775/ref=cm_cr_getr_d_paging_btm_next_2?ie=UTF8&filterByStar=one_star&reviewerType=all_reviews&pageNumber=2#reviews-filter-bar https://www.amazon.com/Introduction-Linear-Algebra-Gilbert-S... ...I of course expect that any popular book will have 1 star reviews, especially by people who had a hard time with the material. Anyone have thoughts whether the Strang book is better as a supplement to other texts in coming up to speed on Linear Algebra? Other book recommendations?

deleted 9y ago

[deleted]

nicolewhite 9y ago

This is a nice intro to linear algebra but doesn't really get into how it's related to deep learning. A good read is Vector, Matrix, and Tensor Derivatives[0], which goes into the linear algebra needed in multi-layer neural networks, specifically during the backpropogation phase. [0] http://cs231n.stanford.edu/vecDerivs.pdf http://cs231n.stanford.edu/vecDerivs.pdf

shogunmike 9y ago
You're correct - it doesn't get into the deep learning aspect yet. This article is in fact the second part of a larger planned series. The idea is to present more depth than a "quick refresher" that seems to be common to a lot of blog posts, but far less material than would be found in a 10-week (or single semester) undergraduate course. Thanks for the Stanford link. I'll check it out.