5 ms·

An Intuitive Derivation of Eigenvectors

209 points by dhruvp 8y ago

wittedhaddock 8y ago
this is awesome tyvm!
lxe 8y ago
I also really liked this explanation: http://setosa.io/ev/eigenvectors-and-eigenvalues/ http://setosa.io/ev/eigenvectors-and-eigenvalues/ You can drag things around and change values -- if you're a visual learner, it really helps grasp things like this.
daeken 8y ago
If you want to get an intuitive, visual understanding of linear algebra -- including eigenvectors/eigenvalues -- 3blue1brown's playlist on the subject is just ... perfect. https://www.youtube.com/watch?v=fNk_zzaMoSs&list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://www.youtube.com/watch?v=fNk_zzaMoSs&list=PLZHQObOWTQ...
- Koshkin 8y ago
  Problem with visual understanding of linear algebra is that it is no understanding at all when it comes to linear spaces over the complex numbers (which are an important tool in quantum mechanics, for example) or infinite-dimensional spaces. Attempts (or the habit) to use the intuition gained when being exposed to an elementary examples often lead to gross misunderstanding and logical errors.
  - eigenloss 8y ago
    It seems like you're suggesting the added complexity of complex numbers makes visual understanding unhelpful; do you care to explain why visualizing complex numbers or multidimensional spaces is so impossible?
quickthrower2 8y ago
Seriously consider buying a domain name for $8 or whatever. You can use it for free with netlify hosting anyway, and then you are in control.
- dhruvp 8y ago
  I know! I’ve just been a little lazy. I’ll get a domain later today.
  - eatonphil 8y ago
    I was confused why/impressed at netlify was writing about eigenvectors in the first place. HN in particular only shows the top-level domain so I couldn't tell it wasn't netlify themselves until clicking around your site. :)
FabHK 8y ago
One way to think of eigenvectors: Your linear map A moves things around, and you aim to characterise the linear map. So, look for lines (through the origin) that are not moved. Those are given by eigenvectors. A point on that line might be moved closer to or further away from the origin (depends on eigenvalue < or > 1), or even flipped to the other side (if eigenvalue < 0), but the line as a whole is mapped to itself.
skywal_l 8y ago
The thing I use to visualize an eigenvector is exactly that. A rotating planet. The eigenvector of the rotation matrix being the axis of rotation. It gets weird when thinking of 2D rotations though... Too complex for me!
- man-and-laptop 8y ago
  The eigenvalues of a 2d rotation matrix are complex numbers. There's no point in trying to visualise the corresponding eigenvectors. You'll need 4 dimensions for that. (The 4 dimensions come from the fact that the eigenvectors of a 2d rotation matrix are elements of ℂ^2, which is topologically equivalent to (ℝ^2)^2, which is equiv to ℝ^4). Personally, my "intuition" is based on analogies with non-complex eigenvectors, and experience solving eigenvector problems algebraically without using pictures. Also, your 3d rotation example actually has three eigenvectors, two of which are complex. You've only found the one that's real.
- dhruvp 8y ago
  OP here: Your approach is pretty on point because Planetary Motion was actually the first recorded motivation for eigenvectors. Cauchy published a paper on planetary motion in 1829 which made use of what we now call Eigenvectors. See here for more info: https://www.maa.org/press/periodicals/convergence/math-origins-eigenvectors-and-eigenvalues https://www.maa.org/press/periodicals/convergence/math-origi...
platform 8y ago
really good explanation. I like it even better than 3blue1brown or visualisations that I had seen. It is better because it really covers every step of the construction process. And offers explanation of why certain thing are not the right construction blocks. The author gives a visual example, for example, of why basic vectors 1,0 -1,0 are bad. The article shows they cannot span the whole space. Those kinds of explanations of 'bad constructions' are difficult to show in visualizations, that show 'good' constructions only. But, yet, in my view, these negative examples, are really helpful to explain the material that otherwise, requires 'intuition'. Not everybody has same intuition, so showing negative examples/impossible constructions, and why those do not work -- is a good way tuning one's intuition. --- On a separate note, I am wondering if such good step by step + counter examples, knowledge presentation -- is a result of author studying at MIT, or a natural trait (or both) ?
- cheez 8y ago
  I think what happens is when you spend enough time thinking about explaining it to others or yourself (for whatever reason), you tend to be able to simplify explanations and find creative ways to explain it. Obviously, it is a mixture of raw talent and something else but there is enough raw talent in this world and not enough something else.
- obastani 8y ago
  You can absolutely visualize why [1, 0] and [-1, 0] is not a basis for R^2! The problem is if you plot them in R^2, then they and the origin all lie on a single line. No matter how you sum copies of these vectors, you can never "escape" this line. Thus, you cannot construct vectors in R^2 that do not lie on this line.
  - platform 8y ago
    Yes, that's exactly how the author of the article is explaining it (including the picture). I was just commenting that it was good that, the author took time to explain this. And those kinds of nuances/counter examples are not always available in explanatory dynamic visualization tools (such as the ones referenced in the comments here [1] ). To be clear, I am not arguing against dynamic visualizations. These are very nice, very very helpful, and very time consuming to build and present on a website. I am just suggesting that showing negative examples is a complimentary, and powerful explanation technique. [1] http://setosa.io/ev/eigenvectors-and-eigenvalues/ http://setosa.io/ev/eigenvectors-and-eigenvalues/
Koshkin 8y ago
There is no shortage of intuitive explanations of various elementary concepts in mathematics (and physics); my personal favorites are by E. Khutoyansky [1]. (Surely enough, there is a video on eigenvectors!) [1] https://m.youtube.com/user/EugeneKhutoryansky https://m.youtube.com/user/EugeneKhutoryansky
jules 8y ago
IMO eigenvectors are easiest to understand in connection with differential equations, and that's also one of their most important applications. If you plot the flow of the equation x' = Ax then the eigenvectors are visually apparent. https://www.wolframalpha.com/input/?i=stream+plot+%7B-5x+%2B+3y,+3x+%2B+8y%7D https://www.wolframalpha.com/input/?i=stream+plot+%7B-5x+%2B... The eigendirections are the directions where the solution moves in a straight line. Not all matrices have (real valued) eigenvectors: https://www.wolframalpha.com/input/?i=stream+plot+%7Bx+%2B+5y,+-x+%2B+y%7D https://www.wolframalpha.com/input/?i=stream+plot+%7Bx+%2B+5...
rodionos 8y ago
I liked the MIT lecture on eigenvectors: https://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebra-fall-2011/least-squares-determinants-and-eigenvalues/eigenvalues-and-eigenvectors/ https://ocw.mit.edu/courses/mathematics/18-06sc-linear-algeb...
billfruit 8y ago
I really think some better name than eigenvectors (and eigenvalues, etc..) should be popularized. I find them to be very obtuse and opaque terms.
- mlevental 8y ago
  eigen in German means owned/proprietary. this has always been a useful mnemonic for me.
BeetleB 8y ago
Good writeup! A tad bit disappointed as I was expecting some new insights, but this is more or less standard material about eigenvectors you would get in a typical university course.
itissid 8y ago
The difference between an exposition in text(like this one) and videos(3blue1brown) is how people prefer to build knowledge and more deeply with how one learns. The quality of both of these is excellent. And one can test what works best like explaining to oneself(or a rubber duck) after reading/viewing the material and how one can recall things.

209 points by dhruvp 8y ago

this is awesome tyvm!

lxe 8y ago

I also really liked this explanation: http://setosa.io/ev/eigenvectors-and-eigenvalues/ http://setosa.io/ev/eigenvectors-and-eigenvalues/ You can drag things around and change values -- if you're a visual learner, it really helps grasp things like this.

daeken 8y ago

If you want to get an intuitive, visual understanding of linear algebra -- including eigenvectors/eigenvalues -- 3blue1brown's playlist on the subject is just ... perfect. https://www.youtube.com/watch?v=fNk_zzaMoSs&list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://www.youtube.com/watch?v=fNk_zzaMoSs&list=PLZHQObOWTQ...

Koshkin 8y ago
Problem with visual understanding of linear algebra is that it is no understanding at all when it comes to linear spaces over the complex numbers (which are an important tool in quantum mechanics, for example) or infinite-dimensional spaces. Attempts (or the habit) to use the intuition gained when being exposed to an elementary examples often lead to gross misunderstanding and logical errors.
- eigenloss 8y ago
  It seems like you're suggesting the added complexity of complex numbers makes visual understanding unhelpful; do you care to explain why visualizing complex numbers or multidimensional spaces is so impossible?

quickthrower2 8y ago

Seriously consider buying a domain name for $8 or whatever. You can use it for free with netlify hosting anyway, and then you are in control.

dhruvp 8y ago
I know! I’ve just been a little lazy. I’ll get a domain later today.
- eatonphil 8y ago
  I was confused why/impressed at netlify was writing about eigenvectors in the first place. HN in particular only shows the top-level domain so I couldn't tell it wasn't netlify themselves until clicking around your site. :)

FabHK 8y ago

One way to think of eigenvectors: Your linear map A moves things around, and you aim to characterise the linear map. So, look for lines (through the origin) that are not moved. Those are given by eigenvectors. A point on that line might be moved closer to or further away from the origin (depends on eigenvalue < or > 1), or even flipped to the other side (if eigenvalue < 0), but the line as a whole is mapped to itself.

skywal_l 8y ago

The thing I use to visualize an eigenvector is exactly that. A rotating planet. The eigenvector of the rotation matrix being the axis of rotation. It gets weird when thinking of 2D rotations though... Too complex for me!

man-and-laptop 8y ago
The eigenvalues of a 2d rotation matrix are complex numbers. There's no point in trying to visualise the corresponding eigenvectors. You'll need 4 dimensions for that. (The 4 dimensions come from the fact that the eigenvectors of a 2d rotation matrix are elements of ℂ^2, which is topologically equivalent to (ℝ^2)^2, which is equiv to ℝ^4). Personally, my "intuition" is based on analogies with non-complex eigenvectors, and experience solving eigenvector problems algebraically without using pictures. Also, your 3d rotation example actually has three eigenvectors, two of which are complex. You've only found the one that's real.
dhruvp 8y ago
OP here: Your approach is pretty on point because Planetary Motion was actually the first recorded motivation for eigenvectors. Cauchy published a paper on planetary motion in 1829 which made use of what we now call Eigenvectors. See here for more info: https://www.maa.org/press/periodicals/convergence/math-origins-eigenvectors-and-eigenvalues https://www.maa.org/press/periodicals/convergence/math-origi...

platform 8y ago

really good explanation. I like it even better than 3blue1brown or visualisations that I had seen. It is better because it really covers every step of the construction process. And offers explanation of why certain thing are not the right construction blocks. The author gives a visual example, for example, of why basic vectors 1,0 -1,0 are bad. The article shows they cannot span the whole space. Those kinds of explanations of 'bad constructions' are difficult to show in visualizations, that show 'good' constructions only. But, yet, in my view, these negative examples, are really helpful to explain the material that otherwise, requires 'intuition'. Not everybody has same intuition, so showing negative examples/impossible constructions, and why those do not work -- is a good way tuning one's intuition. --- On a separate note, I am wondering if such good step by step + counter examples, knowledge presentation -- is a result of author studying at MIT, or a natural trait (or both) ?

cheez 8y ago
I think what happens is when you spend enough time thinking about explaining it to others or yourself (for whatever reason), you tend to be able to simplify explanations and find creative ways to explain it. Obviously, it is a mixture of raw talent and something else but there is enough raw talent in this world and not enough something else.
obastani 8y ago
You can absolutely visualize why [1, 0] and [-1, 0] is not a basis for R^2! The problem is if you plot them in R^2, then they and the origin all lie on a single line. No matter how you sum copies of these vectors, you can never "escape" this line. Thus, you cannot construct vectors in R^2 that do not lie on this line.
- platform 8y ago
  Yes, that's exactly how the author of the article is explaining it (including the picture). I was just commenting that it was good that, the author took time to explain this. And those kinds of nuances/counter examples are not always available in explanatory dynamic visualization tools (such as the ones referenced in the comments here [1] ). To be clear, I am not arguing against dynamic visualizations. These are very nice, very very helpful, and very time consuming to build and present on a website. I am just suggesting that showing negative examples is a complimentary, and powerful explanation technique. [1] http://setosa.io/ev/eigenvectors-and-eigenvalues/ http://setosa.io/ev/eigenvectors-and-eigenvalues/

Koshkin 8y ago

There is no shortage of intuitive explanations of various elementary concepts in mathematics (and physics); my personal favorites are by E. Khutoyansky [1]. (Surely enough, there is a video on eigenvectors!) [1] https://m.youtube.com/user/EugeneKhutoryansky https://m.youtube.com/user/EugeneKhutoryansky

jules 8y ago

IMO eigenvectors are easiest to understand in connection with differential equations, and that's also one of their most important applications. If you plot the flow of the equation x' = Ax then the eigenvectors are visually apparent. https://www.wolframalpha.com/input/?i=stream+plot+%7B-5x+%2B+3y,+3x+%2B+8y%7D https://www.wolframalpha.com/input/?i=stream+plot+%7B-5x+%2B... The eigendirections are the directions where the solution moves in a straight line. Not all matrices have (real valued) eigenvectors: https://www.wolframalpha.com/input/?i=stream+plot+%7Bx+%2B+5y,+-x+%2B+y%7D https://www.wolframalpha.com/input/?i=stream+plot+%7Bx+%2B+5...

rodionos 8y ago

I liked the MIT lecture on eigenvectors: https://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebra-fall-2011/least-squares-determinants-and-eigenvalues/eigenvalues-and-eigenvectors/ https://ocw.mit.edu/courses/mathematics/18-06sc-linear-algeb...

billfruit 8y ago

I really think some better name than eigenvectors (and eigenvalues, etc..) should be popularized. I find them to be very obtuse and opaque terms.

mlevental 8y ago
eigen in German means owned/proprietary. this has always been a useful mnemonic for me.

BeetleB 8y ago

Good writeup! A tad bit disappointed as I was expecting some new insights, but this is more or less standard material about eigenvectors you would get in a typical university course.

itissid 8y ago

The difference between an exposition in text(like this one) and videos(3blue1brown) is how people prefer to build knowledge and more deeply with how one learns. The quality of both of these is excellent. And one can test what works best like explaining to oneself(or a rubber duck) after reading/viewing the material and how one can recall things.