7 ms·

Getting started with linear algebra

226 points by ghosthamlet 8y ago

Brigadirk 8y ago
Thank you for doing this. I'm currently working with deep learning but have never had Linear Algebra in school. Resources like this one, and the 3Blue1Brown video series on YouTube are a really nice way to get started.
- jackallis 8y ago
  3B1B is by far the best resource on LA anywhere in the Web. I might be chastice for saying it but i will go as far to say that 3B1B is better than MIT's LA.
  - harias 8y ago
    3B1B is definitely more useful for applied purposes, but the MIT LA introduces you to the topic formally, and helps you understand research papers.
- mparr4 8y ago
  link: http://www.3blue1brown.com/essence-of-linear-algebra-page/ http://www.3blue1brown.com/essence-of-linear-algebra-page/
isatty 8y ago
Read Introduction to Linear Algebra by Gilbert Strang; should be all you need. There are the MIT OCW lectures that you can use too.
- thanatropism 8y ago
  Obligatory remark that "reading" a math textbook, and even moreso a basic one, is an active task; you can get the intuitions from youtube videos of the 3Blue1Brown variety, but to really learn you need to follow the text closely, put question marks on passages you're not 120% comfortable with, do the proofs that are left to the reader and the recommended exercises. If you're super comfortable with the material, you should do the more challenging exercises, usually marked with asterisks. Basically you need a pencil, a lot of scratch paper and a capable waste basket; or a whiteboard all to yourself (but I get tired from being on my feet for hours on end). ---- E: My litmus test is the Gershgorin circle theorem. Every so often I check if I can still prove it, and if I do I try to review the surrounding matrix. I will literally vote against hiring anyone who can't at least handwave why that's true.
  - deleted 8y ago
    [deleted]
  - davidmr 8y ago
    Purely out of curiosity, what jobs is your employer trying to fill that an intuitive understanding of said theorem is required? I took a lot of linear algebra and I can’t recall having covered it.
  - WoodenChair 8y ago
    >My litmus test is the Gershgorin circle theorem. Every so often I check if I can still prove it, and if I do I try to review the surrounding matrix. I will literally vote against hiring anyone who can't at least handwave why that's true. Interesting comment with no context—i.e. not hire someone for what? I don't know if I would ever not hire someone for not understanding a math problem (unless the job was heavily math related in the sphere of the problem). I would definitely not hire someone who communicates as ineffectively as you did in this comment.
    - tjr 8y ago
      Reminds me of: http://philip.greenspun.com/humor/mcdonalds-joke http://philip.greenspun.com/humor/mcdonalds-joke
  - jnbiche 8y ago
    > I will literally vote against hiring anyone who can't at least handwave why that's true. I hope you're at least hiring for a machine learning engineer and not a general software engineer to build CRUD apps.
- internet555 8y ago
  Certainly you would want a more abstract follow up for many cases. Even engineering or computer science students may have to learn about e.g. inner product spaces and orthogonal polynomials for their numerics class.
geokon 8y ago
Where is the text? Or is this an advertisement for a book? Since this might get the right eyeballs here: 1. Does anyone have a good intuitive explanation for the meaning of the transpose operation? It seems to tie into a central symmetry between columns and rows (or equations and variables) that I'm having trouble articulating for myself 2. Why does every linear algebra book have a chapter on norms? It's always so incredibly boring and esoteric and I still haven't hit a situation where it's strictly necessary. The 2-norm seem adequate for most situations and comes out naturally from the least squares problem PS: Strang is great. Meyer's "Matrix analysis and applied linear algebra" is even better (but more advanced)
- weavie 8y ago
  Click on the numbered headings. They don't look like links, but take you to the text for that section.
- MichaelAO 8y ago
  Check out Better Explained for a good intuitive guide to linear algebra (among other things): https://betterexplained.com/articles/matrix-multiplication/ https://betterexplained.com/articles/matrix-multiplication/
- internet555 8y ago
  For one example the notion of a generalized inner product and its induced norm is important for the derivation of conjugate gradient algorithm. My smarter friends have told me that anything involving cost functions and covariance matrices (as might arise in a log likelihood thing for a multivariate normal random variable) will probably end up with weighted norms as well, but I have likely mangled this. So to mangle things further I think that this is very important to many fields eg control or statistics
- currymj 8y ago
  i mean, probably the sexiest application of linear algebra is machine learning and different exotic norms are very important there. sometimes you even have to learn a norm from data (metric learning). at the very least i feel like it’s important to understand 0, 1, 2, and infinity norm.
vladsanchez 8y ago
Hadrien's book is so engaging I couldn't stop reading. It's superbly illustrated. It steadily and seamlessly progresses through each topic that you don't realize you're actually learning 'deep learning'. I'm sure this book will get lots of praise pretty soon. Thanks for sharing it ghosthamlet. Thanks for writing it Hadrien.
deleted 8y ago
[deleted]
myWindoonn 8y ago
But https://graphicallinearalgebra.net/ https://graphicallinearalgebra.net/ is available for free and uses easier-to-read graphical syntax. Why prefer this book instead?
- rhcom2 8y ago
  I've been doing the Khan Academy linear algebra course which is good too.
- Jtsummers 8y ago
  The original linked text is probably better for a linear algebra learner if they want to apply it (especially if they want to apply it earlier rather than later). Your linked text looks (I've read the first 5 parts) to be a good introduction but at a higher level of abstraction (category theory and abstract algebra). In fact, I think it may present a good motivating case for understanding that level of higher abstraction (while it's ostensibly about linear algebra, the ideas transfer with some adjustments to other algebras). It could also use some polish. It's awkward to be starting a new "episode" and find that it's two pages of off-topic discussion (presenting the case for the blog, which would serve better in an introduction than in the middle of the material).
neurotrace 8y ago
Looks like some great information but you need to change the way you style the links to the different chapters. The links look like ordinary text and I had no idea that I could click them until I accidentally hovered over one.
- hadrienj 8y ago
  Thanks! It was a style error. I just fixed it!
Koshkin 8y ago
> A tensor is a n-dimensional array with n>2 This is just one example of why it is usually better to learn stuff from more authoritative sources. (On the other hand, I can’t help but recommend this amazing video: https://m.youtube.com/watch?v=CliW7kSxxWU. https://m.youtube.com/watch?v=CliW7kSxxWU.)

226 points by ghosthamlet 8y ago

Thank you for doing this. I'm currently working with deep learning but have never had Linear Algebra in school. Resources like this one, and the 3Blue1Brown video series on YouTube are a really nice way to get started.

jackallis 8y ago
3B1B is by far the best resource on LA anywhere in the Web. I might be chastice for saying it but i will go as far to say that 3B1B is better than MIT's LA.
- harias 8y ago
  3B1B is definitely more useful for applied purposes, but the MIT LA introduces you to the topic formally, and helps you understand research papers.
mparr4 8y ago
link: http://www.3blue1brown.com/essence-of-linear-algebra-page/ http://www.3blue1brown.com/essence-of-linear-algebra-page/

isatty 8y ago

Read Introduction to Linear Algebra by Gilbert Strang; should be all you need. There are the MIT OCW lectures that you can use too.

thanatropism 8y ago
Obligatory remark that "reading" a math textbook, and even moreso a basic one, is an active task; you can get the intuitions from youtube videos of the 3Blue1Brown variety, but to really learn you need to follow the text closely, put question marks on passages you're not 120% comfortable with, do the proofs that are left to the reader and the recommended exercises. If you're super comfortable with the material, you should do the more challenging exercises, usually marked with asterisks. Basically you need a pencil, a lot of scratch paper and a capable waste basket; or a whiteboard all to yourself (but I get tired from being on my feet for hours on end). ---- E: My litmus test is the Gershgorin circle theorem. Every so often I check if I can still prove it, and if I do I try to review the surrounding matrix. I will literally vote against hiring anyone who can't at least handwave why that's true.
- deleted 8y ago
  [deleted]
- davidmr 8y ago
  Purely out of curiosity, what jobs is your employer trying to fill that an intuitive understanding of said theorem is required? I took a lot of linear algebra and I can’t recall having covered it.
- WoodenChair 8y ago
  >My litmus test is the Gershgorin circle theorem. Every so often I check if I can still prove it, and if I do I try to review the surrounding matrix. I will literally vote against hiring anyone who can't at least handwave why that's true. Interesting comment with no context—i.e. not hire someone for what? I don't know if I would ever not hire someone for not understanding a math problem (unless the job was heavily math related in the sphere of the problem). I would definitely not hire someone who communicates as ineffectively as you did in this comment.
  - tjr 8y ago
    Reminds me of: http://philip.greenspun.com/humor/mcdonalds-joke http://philip.greenspun.com/humor/mcdonalds-joke
- jnbiche 8y ago
  > I will literally vote against hiring anyone who can't at least handwave why that's true. I hope you're at least hiring for a machine learning engineer and not a general software engineer to build CRUD apps.
internet555 8y ago
Certainly you would want a more abstract follow up for many cases. Even engineering or computer science students may have to learn about e.g. inner product spaces and orthogonal polynomials for their numerics class.

geokon 8y ago

Where is the text? Or is this an advertisement for a book? Since this might get the right eyeballs here: 1. Does anyone have a good intuitive explanation for the meaning of the transpose operation? It seems to tie into a central symmetry between columns and rows (or equations and variables) that I'm having trouble articulating for myself 2. Why does every linear algebra book have a chapter on norms? It's always so incredibly boring and esoteric and I still haven't hit a situation where it's strictly necessary. The 2-norm seem adequate for most situations and comes out naturally from the least squares problem PS: Strang is great. Meyer's "Matrix analysis and applied linear algebra" is even better (but more advanced)

weavie 8y ago
Click on the numbered headings. They don't look like links, but take you to the text for that section.
MichaelAO 8y ago
Check out Better Explained for a good intuitive guide to linear algebra (among other things): https://betterexplained.com/articles/matrix-multiplication/ https://betterexplained.com/articles/matrix-multiplication/
internet555 8y ago
For one example the notion of a generalized inner product and its induced norm is important for the derivation of conjugate gradient algorithm. My smarter friends have told me that anything involving cost functions and covariance matrices (as might arise in a log likelihood thing for a multivariate normal random variable) will probably end up with weighted norms as well, but I have likely mangled this. So to mangle things further I think that this is very important to many fields eg control or statistics
currymj 8y ago
i mean, probably the sexiest application of linear algebra is machine learning and different exotic norms are very important there. sometimes you even have to learn a norm from data (metric learning). at the very least i feel like it’s important to understand 0, 1, 2, and infinity norm.

vladsanchez 8y ago

Hadrien's book is so engaging I couldn't stop reading. It's superbly illustrated. It steadily and seamlessly progresses through each topic that you don't realize you're actually learning 'deep learning'. I'm sure this book will get lots of praise pretty soon. Thanks for sharing it ghosthamlet. Thanks for writing it Hadrien.

deleted 8y ago

[deleted]

myWindoonn 8y ago

But https://graphicallinearalgebra.net/ https://graphicallinearalgebra.net/ is available for free and uses easier-to-read graphical syntax. Why prefer this book instead?

rhcom2 8y ago
I've been doing the Khan Academy linear algebra course which is good too.
Jtsummers 8y ago
The original linked text is probably better for a linear algebra learner if they want to apply it (especially if they want to apply it earlier rather than later). Your linked text looks (I've read the first 5 parts) to be a good introduction but at a higher level of abstraction (category theory and abstract algebra). In fact, I think it may present a good motivating case for understanding that level of higher abstraction (while it's ostensibly about linear algebra, the ideas transfer with some adjustments to other algebras). It could also use some polish. It's awkward to be starting a new "episode" and find that it's two pages of off-topic discussion (presenting the case for the blog, which would serve better in an introduction than in the middle of the material).

neurotrace 8y ago

Looks like some great information but you need to change the way you style the links to the different chapters. The links look like ordinary text and I had no idea that I could click them until I accidentally hovered over one.

hadrienj 8y ago
Thanks! It was a style error. I just fixed it!

Koshkin 8y ago

> A tensor is a n-dimensional array with n>2 This is just one example of why it is usually better to learn stuff from more authoritative sources. (On the other hand, I can’t help but recommend this amazing video: https://m.youtube.com/watch?v=CliW7kSxxWU. https://m.youtube.com/watch?v=CliW7kSxxWU.)