8 ms·

Interactive Linear Algebra (2019)

365 points by natemcintosh 5y ago

TrackerFF 5y ago
I really wish we'd have these kind of tools when I took Linear Algebra (or many other math and engineering courses, for that mater). When I took it, it was purely proofs up and down on the blackboard, zero visualizations.
- chobytes 5y ago
  I think the trouble with LA education is that LA is just so ubiquitous and can look very different dependent on context. No style is going to suit everyones needs.
- Jtsummers 5y ago
  When you took it isn't really the issue as much as where you took it. Some schools and college majors choose different ways of teaching. I liked my exposure which was over 3 courses: Calc 2 had an embedded LA course; a 2xx level computational-focused LA course (much expanded version of the 2 or 3 week Calc 2 version); a 4xx math major only proof-focused LA course. Most students just needed the brief intro. A substantial fraction (but not the majority) got the 2xx computational form. A very small number of us took the 4xx version, and since (by then) you understood the applications there was little need to focus on computations since the interesting (to the course) bit was the higher level understanding of LA.
iainctduncan 5y ago
I am excited to try this, thanks for posting!
nicomeemes 5y ago
This is one of my favorite resources for learning about Linear Algebra. Helped me immensely when I took it last spring.
criddell 5y ago
It would be nice if the items in the left column on the index page [1] were links to the context location. [1]: http://textbooks.math.gatech.edu/ila/index-1.html http://textbooks.math.gatech.edu/ila/index-1.html
scythmic_waves 5y ago
I've only just skimmed through this. And it's a subject I already know so I can't tell if it's actually a good resource. However, my initial impression is that I love it. I think that textbooks, math textbooks in particular, are an example where print publishing does a disservice. (I'm counting PDFs here too.) By having to lay everything out in print form, you have to clutter up your explanations with examples and footnotes that take up physical room. Here, the examples are toggle-able. If I _want_ to explore an example, I can. But I don't need to. This kind of thing is especially helpful when reviewing content for, say, a test rather than learning it for the first time. Also finding things in textbooks is a real pain. It's difficult to index things in a helpful way, so you just have these counting schemes in LaTeX that increment for every definition, theorem, etc. I'd love to be able to tag things then search the tags. And that says nothing for when you want to explain something that's difficult with static images. Being able to interact with animations by zooming, panning, pausing, slowing down, speeding up, etc. is a boon. (I don't think I actually saw an example of a non-static image here, but I think my point still stands.) All in all, I'd love to see more interactive textbooks. We've got this really expressive kind of document via the web. I think we should be taking advantage of it more.
- JadeNB 5y ago
  > I think that textbooks, math textbooks in particular, are an example where print publishing does a disservice. (I'm counting PDFs here too.) By having to lay everything out in print form, you have to clutter up your explanations with examples and footnotes that take up physical room. Here, the examples are toggle-able. If I _want_ to explore an example, I can. But I don't need to. This kind of thing is especially helpful when reviewing content for, say, a test rather than learning it for the first time. This sounds so much like a meeting of Lamport's idea of how to structure a proof (https://lamport.azurewebsites.net/pubs/proof.pdf https://lamport.azurewebsites.net/pubs/proof.pdf) and the Stacks project (https://stacks.math.columbia.edu https://stacks.math.columbia.edu).
  - scythmic_waves 5y ago
    I had never seen Lamport's idea of structured proofs. I just read the linked paper and I love it.
ChrisArchitect 5y ago
Previous discussion: 2 years ago https://news.ycombinator.com/item?id=21628449 https://news.ycombinator.com/item?id=21628449
ogogmad 5y ago
I wrote an entry on Wikipedia on visualizing the QR algorithm: https://en.wikipedia.org/wiki/QR_algorithm https://en.wikipedia.org/wiki/QR_algorithm The visualization helped me spot an unstable fixed point and understand the behaviour of the algorithm near eigenvalue clashes. The behaviour's quite sophisticated. I think I wrote this there, but here goes again. The idea is that a positive-definite symmetric matrix can be visualized as an ellipse. This follows from the spectral theorem. Each iteration of the QR algorithm causes the ellipse to fall towards the x-axis, as if under the influence of gravity. The unstable fixed point corresponds to when the ellipse is standing up precariously, unable to fall in either direction. If you tilt it by just a bit, it will fall over (so the fixed point is unstable). The case when the ellipse is nearly circular (corresponding to near eigenvalue clashes) causes the ellipse to fall over slowly. I think this also makes physical sense, if you think of it being under the influence of gravity. If you think of this near-circle as being a matrix, then this matrix is nearly equal to a scalar multiple of the identity matrix, so its eigenvalues are essentially known. The fact that the ellipse falls very slowly implies that the eigenvectors are unstable near eigenvalue clashes, but the eigenvalues are easy to find. Note: The issues surrounding the unstable fixed point can be fixed using Wilkinson shifts. This makes each iteration into a discontinuous function, allowing all the fixed points to be stable. The issue surrounding instability of the eigenvectors near eigenvalue clashes cannot be fixed, as it's intrinsic to eigendecomposition (even of symmetric matrices). The latter difficulties can be dodged by slightly perturbing the matrix, but the resulting eigenvectors can be very different from the eigenvectors of the unperturbed matrix.
- gugagore 5y ago
  > a positive-definite symmetric matrix can be visualized as an ellipse. It is clear that you mean the ellipsoid as a set of points {x | x^T * A * x = 1} (or some other constant). There is another way in which all square matrices define an ellipsoid based on how the matrix transforms a unit sphere: {A*x | x^T * x = 1} (different matrices can map to the same ellipse here, however). I always like to clarify which one. (I unfortunately do not have intuition for the QR algorithm, but I am distracted by the description of an ellipse falling, as if it's rolling against the x-axis)
whytaka 5y ago
Being aware of the broad applications for Linear Algebra in engineering, I'm very eager to go through some Linear Algebra course on my own. But my problem is that I can't just learn from a text, even if it's interactive. I need to apply it to something. What are some fun projects that uses LA for an individual? I'm thinking about things like generative art, if anyone knows of any artists that inspire them.
- gugagore 5y ago
  One possible perspective: thinking of images as a vector space, and thinking of image operations like blurring, sharpening, and even any spatial transformation as linear operators (representable as matrices). This perspective is most straightforward with a 1-dimensional domain, like an audio signal, or the temperature along a thin rod, because there's a clear way to organize all the values in a vector, e.g. all the audio samples are just in sequence. So if that's appealing to you at all, I would recommend that first. You can do filtering (like with an "equalizer") and add effects like echo/reverb as linear operations. I think that it is particularly fun to think of some ground-truth signal that is corrupted by an echo in a room and a microphone that attenuates some frequencies, and trying to undo that corruption.
- chongli 5y ago
  You could try making a simple software-rendered 3D game from scratch. Basic 3D rendering is pretty much all linear algebra computations involving matrices and vectors.
- ampgt 5y ago
  Check out “eigenfaces.” It’s straight forward and works reasonably well. There are a lot of interesting applications of PCA and SVD. Signal and noise separation, machine learning, the list goes on.
nla 5y ago
FYI, FB blocks this as "violating community standards" when you try to post it there. And also, it's awesome!
testkitchen 5y ago
This is great! This is a nice direction to go. Also, I don't mind if the creators would like to charge a small fee to use the book.
master_yoda_1 5y ago
It's confusing why ml beginners are obsessed over linear algebra. The subject is very hard and need only for advanced ml model. why not just study calculus and understand it?
sonograph 5y ago
I hope that GATech one day offers an affordable online math MSc program, like their CS program
ampgt 5y ago
Cool to see an article on the front page of HN from my alma mater :)
screye 5y ago
For those interested in these kinds of interractive math experiences, I have been keeping track of them for a while. Here is my list so far: • https://www.intmath.com/ https://www.intmath.com/ - Interactive Mathematics Learn math while you play with it • http://worrydream.com/LadderOfAbstraction/ http://worrydream.com/LadderOfAbstraction/ - up and down the ladder of abstraction • https://betterexplained.com/ https://betterexplained.com/ - Intuitive guides to various things in math • https://www.math3ma.com/blog/matrices-probability-graphs https://www.math3ma.com/blog/matrices-probability-graphs - Viewing Matrices & Probability as Graphs • http://immersivemath.com/ila/index.html http://immersivemath.com/ila/index.html - immersive linear alg
- westurner 5y ago
  https://github.com/topics/linear-algebra?l=jupyter+notebook https://github.com/topics/linear-algebra?l=jupyter+notebook lists "Computational Linear Algebra for Coders" https://github.com/fastai/numerical-linear-algebra https://github.com/fastai/numerical-linear-algebra "site:GitHub.com inurl:awesome linear algebra jupyter" lists a few awesome lists with interactive linear algebra resources: https://www.google.com/search?q=site%3Agithub.com+inurl%3Aawesome+linear+algebra+jupyter https://www.google.com/search?q=site%3Agithub.com+inurl%3Aaw... 3blue1brown's "Essence of linear algebra" playlist has some excellent tutorials with intuition-building visualizations built with manim: https://youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFit... https://github.com/ManimCommunity/manim https://github.com/ManimCommunity/manim
- app4soft 5y ago
  microMathematics Plus[0,1,2] (FLOSS Android app) > With microMathematics Plus, not only can you perform mathematical calculations in naturally readable form but you can also create and manage your own collection of interactive formulas! [0] https://github.com/mkulesh/microMathematics https://github.com/mkulesh/microMathematics [1] https://f-droid.org/packages/com.mkulesh.micromath.plus https://f-droid.org/packages/com.mkulesh.micromath.plus [2] https://play.google.com/store/apps/details?id=com.mkulesh.micromath.plus https://play.google.com/store/apps/details?id=com.mkulesh.mi...
jdlyga 5y ago
Georgia Tech! Anyone else just finish taking Graduate Algorithms? Hope you passed.
- gautamcgoel 5y ago
  Hah! I took that course with Vijay Vazirani almost a decade ago.

365 points by natemcintosh 5y ago

I really wish we'd have these kind of tools when I took Linear Algebra (or many other math and engineering courses, for that mater). When I took it, it was purely proofs up and down on the blackboard, zero visualizations.

chobytes 5y ago
I think the trouble with LA education is that LA is just so ubiquitous and can look very different dependent on context. No style is going to suit everyones needs.
Jtsummers 5y ago
When you took it isn't really the issue as much as where you took it. Some schools and college majors choose different ways of teaching. I liked my exposure which was over 3 courses: Calc 2 had an embedded LA course; a 2xx level computational-focused LA course (much expanded version of the 2 or 3 week Calc 2 version); a 4xx math major only proof-focused LA course. Most students just needed the brief intro. A substantial fraction (but not the majority) got the 2xx computational form. A very small number of us took the 4xx version, and since (by then) you understood the applications there was little need to focus on computations since the interesting (to the course) bit was the higher level understanding of LA.

iainctduncan 5y ago

I am excited to try this, thanks for posting!

nicomeemes 5y ago

This is one of my favorite resources for learning about Linear Algebra. Helped me immensely when I took it last spring.

criddell 5y ago

It would be nice if the items in the left column on the index page [1] were links to the context location. [1]: http://textbooks.math.gatech.edu/ila/index-1.html http://textbooks.math.gatech.edu/ila/index-1.html

scythmic_waves 5y ago

I've only just skimmed through this. And it's a subject I already know so I can't tell if it's actually a good resource. However, my initial impression is that I love it. I think that textbooks, math textbooks in particular, are an example where print publishing does a disservice. (I'm counting PDFs here too.) By having to lay everything out in print form, you have to clutter up your explanations with examples and footnotes that take up physical room. Here, the examples are toggle-able. If I _want_ to explore an example, I can. But I don't need to. This kind of thing is especially helpful when reviewing content for, say, a test rather than learning it for the first time. Also finding things in textbooks is a real pain. It's difficult to index things in a helpful way, so you just have these counting schemes in LaTeX that increment for every definition, theorem, etc. I'd love to be able to tag things then search the tags. And that says nothing for when you want to explain something that's difficult with static images. Being able to interact with animations by zooming, panning, pausing, slowing down, speeding up, etc. is a boon. (I don't think I actually saw an example of a non-static image here, but I think my point still stands.) All in all, I'd love to see more interactive textbooks. We've got this really expressive kind of document via the web. I think we should be taking advantage of it more.

JadeNB 5y ago
> I think that textbooks, math textbooks in particular, are an example where print publishing does a disservice. (I'm counting PDFs here too.) By having to lay everything out in print form, you have to clutter up your explanations with examples and footnotes that take up physical room. Here, the examples are toggle-able. If I _want_ to explore an example, I can. But I don't need to. This kind of thing is especially helpful when reviewing content for, say, a test rather than learning it for the first time. This sounds so much like a meeting of Lamport's idea of how to structure a proof (https://lamport.azurewebsites.net/pubs/proof.pdf https://lamport.azurewebsites.net/pubs/proof.pdf) and the Stacks project (https://stacks.math.columbia.edu https://stacks.math.columbia.edu).
- scythmic_waves 5y ago
  I had never seen Lamport's idea of structured proofs. I just read the linked paper and I love it.

ChrisArchitect 5y ago

Previous discussion: 2 years ago https://news.ycombinator.com/item?id=21628449 https://news.ycombinator.com/item?id=21628449

ogogmad 5y ago

I wrote an entry on Wikipedia on visualizing the QR algorithm: https://en.wikipedia.org/wiki/QR_algorithm https://en.wikipedia.org/wiki/QR_algorithm The visualization helped me spot an unstable fixed point and understand the behaviour of the algorithm near eigenvalue clashes. The behaviour's quite sophisticated. I think I wrote this there, but here goes again. The idea is that a positive-definite symmetric matrix can be visualized as an ellipse. This follows from the spectral theorem. Each iteration of the QR algorithm causes the ellipse to fall towards the x-axis, as if under the influence of gravity. The unstable fixed point corresponds to when the ellipse is standing up precariously, unable to fall in either direction. If you tilt it by just a bit, it will fall over (so the fixed point is unstable). The case when the ellipse is nearly circular (corresponding to near eigenvalue clashes) causes the ellipse to fall over slowly. I think this also makes physical sense, if you think of it being under the influence of gravity. If you think of this near-circle as being a matrix, then this matrix is nearly equal to a scalar multiple of the identity matrix, so its eigenvalues are essentially known. The fact that the ellipse falls very slowly implies that the eigenvectors are unstable near eigenvalue clashes, but the eigenvalues are easy to find. Note: The issues surrounding the unstable fixed point can be fixed using Wilkinson shifts. This makes each iteration into a discontinuous function, allowing all the fixed points to be stable. The issue surrounding instability of the eigenvectors near eigenvalue clashes cannot be fixed, as it's intrinsic to eigendecomposition (even of symmetric matrices). The latter difficulties can be dodged by slightly perturbing the matrix, but the resulting eigenvectors can be very different from the eigenvectors of the unperturbed matrix.

gugagore 5y ago
> a positive-definite symmetric matrix can be visualized as an ellipse. It is clear that you mean the ellipsoid as a set of points {x | x^T * A * x = 1} (or some other constant). There is another way in which all square matrices define an ellipsoid based on how the matrix transforms a unit sphere: {A*x | x^T * x = 1} (different matrices can map to the same ellipse here, however). I always like to clarify which one. (I unfortunately do not have intuition for the QR algorithm, but I am distracted by the description of an ellipse falling, as if it's rolling against the x-axis)

whytaka 5y ago

Being aware of the broad applications for Linear Algebra in engineering, I'm very eager to go through some Linear Algebra course on my own. But my problem is that I can't just learn from a text, even if it's interactive. I need to apply it to something. What are some fun projects that uses LA for an individual? I'm thinking about things like generative art, if anyone knows of any artists that inspire them.

gugagore 5y ago
One possible perspective: thinking of images as a vector space, and thinking of image operations like blurring, sharpening, and even any spatial transformation as linear operators (representable as matrices). This perspective is most straightforward with a 1-dimensional domain, like an audio signal, or the temperature along a thin rod, because there's a clear way to organize all the values in a vector, e.g. all the audio samples are just in sequence. So if that's appealing to you at all, I would recommend that first. You can do filtering (like with an "equalizer") and add effects like echo/reverb as linear operations. I think that it is particularly fun to think of some ground-truth signal that is corrupted by an echo in a room and a microphone that attenuates some frequencies, and trying to undo that corruption.
chongli 5y ago
You could try making a simple software-rendered 3D game from scratch. Basic 3D rendering is pretty much all linear algebra computations involving matrices and vectors.
ampgt 5y ago
Check out “eigenfaces.” It’s straight forward and works reasonably well. There are a lot of interesting applications of PCA and SVD. Signal and noise separation, machine learning, the list goes on.

nla 5y ago

FYI, FB blocks this as "violating community standards" when you try to post it there. And also, it's awesome!

testkitchen 5y ago

This is great! This is a nice direction to go. Also, I don't mind if the creators would like to charge a small fee to use the book.

master_yoda_1 5y ago

It's confusing why ml beginners are obsessed over linear algebra. The subject is very hard and need only for advanced ml model. why not just study calculus and understand it?

sonograph 5y ago

I hope that GATech one day offers an affordable online math MSc program, like their CS program

ampgt 5y ago

Cool to see an article on the front page of HN from my alma mater :)

screye 5y ago

For those interested in these kinds of interractive math experiences, I have been keeping track of them for a while. Here is my list so far: • https://www.intmath.com/ https://www.intmath.com/ - Interactive Mathematics Learn math while you play with it • http://worrydream.com/LadderOfAbstraction/ http://worrydream.com/LadderOfAbstraction/ - up and down the ladder of abstraction • https://betterexplained.com/ https://betterexplained.com/ - Intuitive guides to various things in math • https://www.math3ma.com/blog/matrices-probability-graphs https://www.math3ma.com/blog/matrices-probability-graphs - Viewing Matrices & Probability as Graphs • http://immersivemath.com/ila/index.html http://immersivemath.com/ila/index.html - immersive linear alg

westurner 5y ago
https://github.com/topics/linear-algebra?l=jupyter+notebook https://github.com/topics/linear-algebra?l=jupyter+notebook lists "Computational Linear Algebra for Coders" https://github.com/fastai/numerical-linear-algebra https://github.com/fastai/numerical-linear-algebra "site:GitHub.com inurl:awesome linear algebra jupyter" lists a few awesome lists with interactive linear algebra resources: https://www.google.com/search?q=site%3Agithub.com+inurl%3Aawesome+linear+algebra+jupyter https://www.google.com/search?q=site%3Agithub.com+inurl%3Aaw... 3blue1brown's "Essence of linear algebra" playlist has some excellent tutorials with intuition-building visualizations built with manim: https://youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFit... https://github.com/ManimCommunity/manim https://github.com/ManimCommunity/manim
app4soft 5y ago
microMathematics Plus[0,1,2] (FLOSS Android app) > With microMathematics Plus, not only can you perform mathematical calculations in naturally readable form but you can also create and manage your own collection of interactive formulas! [0] https://github.com/mkulesh/microMathematics https://github.com/mkulesh/microMathematics [1] https://f-droid.org/packages/com.mkulesh.micromath.plus https://f-droid.org/packages/com.mkulesh.micromath.plus [2] https://play.google.com/store/apps/details?id=com.mkulesh.micromath.plus https://play.google.com/store/apps/details?id=com.mkulesh.mi...

jdlyga 5y ago

Georgia Tech! Anyone else just finish taking Graduate Algorithms? Hope you passed.

gautamcgoel 5y ago
Hah! I took that course with Vijay Vazirani almost a decade ago.