4 ms·

The A in the title is for AWESOME. I have been writing about linear algebra almost non-stop for the past two months to organize the best possible "playlist" of

by ivan_ah 13y ago

The A in the title is for AWESOME. I have been writing about linear algebra almost non-stop for the past two months to organize the best possible "playlist" of topics that cover a linear algebra course. It's a hard problem, this linearization of a graph of dependencies into a stream of .tex Prof. Simoncelli has a very good geometrical take on the problem. He manages to cover all the essential topics in 10 pages. Geometrically speaking, vectors = arrows and matrices = transformations on arrows.

yxhuvud 13y ago
All the essential topics? I miss eigenvalues (and probably determinants as a stepping stone to that). Dunno if the most pedagogical approach would be before or after SVD though.
- olympus 13y ago
  He touches on it at the end with Singular Value Decomposition, but didn't mention that you can use SVD to get eigenvectors/values. He also doesn't cover how to do SVD. This could definitely be covered more in depth, but maybe not in a single semester undergraduate course. I think a follow on set of notes that starts with eigenvectors/values and continues into the more fun parts of linear algebra would be very useful.
- j2kun 13y ago
  Definitely don't need determinants to learn everything you need to know about eigenvalues.
- ivan_ah 13y ago
  > I miss eigenvalues They're not missing just judiciously omitted. Actually seeing SVD presented in an introduction is what inspired me to post this. The SVD approach works for an matrix A in R^{m x n} A = UΣV^T But if A is square and you allow the singular values to be +/-, then you can find a single matrix U that serves as a left-basis and a right-basis for A. Hence we obtain the eigendecomposition: A = UΛU^T So SVD is the mother ship, and eigenstuff is a special case.
- icegreentea 13y ago
  At least from a non-software engineering context, you're usually taught eigenvalues first as they are quite useful for a lot different problems... basically all linear systems. At least the way our curriculum was arranged, in first year had a 'simple' linear algebra course that consisted of solving systems of linear equations (like Gaussian Elimination), and 'simple' things with matrices and vectors, which then ended in this rush to teach eigen-decomposition, which we pretty much just learned by rote. I don't believe anyone really had a feel for what we were actually doing. But the flip side was that we learned eigen-decomposition in time for all the 'fun' applications of it, and by the time our far more rigorous linear algebra course occurred in 3rd year (which ended with SVD), when we went over eigen-decomp again, it was like a brilliant 'OH, I GET ITT' moment for everyone in the class.
marchdown 13y ago
I would love to see your take on this dependency graph, no matter how subjective and incomplete it turns out to be.
- ivan_ah 13y ago
  {arithmetic} | {vecops} = {Vector operations} {dotproduct} {arithmetic} | {RREF} = {Reduced row echelon form} {arithmetic} | {patience} | | {dotproduct} | | | {matrices} {Mops} = {Matrix operations} {Meqns} = {Matrix equations} {Mmult} = {Matrix multiplication} {Mdet} = {Determinants} {Minv} = {Matrix inverse} {y=mx+b} | {linear} = {Linearity} {plane} = {Lines and planes} {dotproduct} | {proj} = {Projections} {basis} = {Coordinate projections} {vectors} | {VS1} = {Vector spaces} {VS2} = {Abstract vector spaces} // knowledge buzz moment | | {dotproduct} | | {IPS} = {Abstract inner product spaces} {y=mx} | {Mmult} | | {function f:R->R} | | | {linT} = {Linear transformations} {VSRREF} = {Vector space techniques for matrices} {TasM} = {Finding matrix representations} {basis2} = {Change of basis for matrices} {Mmult} | {linT} | | {eigV} = {Eigenvalues and eigenvectors} {Mdecomp}= {Matrix decompositions} | {specM} = {Special types of matrices} {VS1} |{plane} ||{eigV} |||{linT} |||||{Meqns} ||||||{Mmult} ||||||||{Mdet} ||||||||| {invMthm} = {Invertible matrix theorem} {proj} | {GSorth} = {Gram--Schmidt orthogonalization} {RREF} | {LP} = {Linear programming} {Mmult} |{Meqn} ||{RREF} ||| {solve} = {Solving systems of equations} {LSaprx} = {Least squares approximate solution} {ML} = {Machine learning} {LAoverZ_q}= {Linear algebra over a finite field} ||| ||{ECCs} = {Error correcting codes} |{crypto} = {Cryptography} {netcode}= {Network coding} {trig} |{taylor} ||{AVS} |||{IPS} ||||| {fourier}= {Fourier series} {fourier} | {prob} = {Probability density} | | {LAoverC}= {Linear algebra with complex numbers} | | | {quantum}= {Quantum mechanics} UPDATE: Okay so if anyone is interested, I just put up a pre-release of the book https://gum.co/noBSLA https://gum.co/noBSLA

by ivan_ah 13y ago

yxhuvud 13y ago

All the essential topics? I miss eigenvalues (and probably determinants as a stepping stone to that). Dunno if the most pedagogical approach would be before or after SVD though.

olympus 13y ago
He touches on it at the end with Singular Value Decomposition, but didn't mention that you can use SVD to get eigenvectors/values. He also doesn't cover how to do SVD. This could definitely be covered more in depth, but maybe not in a single semester undergraduate course. I think a follow on set of notes that starts with eigenvectors/values and continues into the more fun parts of linear algebra would be very useful.
j2kun 13y ago
Definitely don't need determinants to learn everything you need to know about eigenvalues.
ivan_ah 13y ago
> I miss eigenvalues They're not missing just judiciously omitted. Actually seeing SVD presented in an introduction is what inspired me to post this. The SVD approach works for an matrix A in R^{m x n} A = UΣV^T But if A is square and you allow the singular values to be +/-, then you can find a single matrix U that serves as a left-basis and a right-basis for A. Hence we obtain the eigendecomposition: A = UΛU^T So SVD is the mother ship, and eigenstuff is a special case.
icegreentea 13y ago
At least from a non-software engineering context, you're usually taught eigenvalues first as they are quite useful for a lot different problems... basically all linear systems. At least the way our curriculum was arranged, in first year had a 'simple' linear algebra course that consisted of solving systems of linear equations (like Gaussian Elimination), and 'simple' things with matrices and vectors, which then ended in this rush to teach eigen-decomposition, which we pretty much just learned by rote. I don't believe anyone really had a feel for what we were actually doing. But the flip side was that we learned eigen-decomposition in time for all the 'fun' applications of it, and by the time our far more rigorous linear algebra course occurred in 3rd year (which ended with SVD), when we went over eigen-decomp again, it was like a brilliant 'OH, I GET ITT' moment for everyone in the class.

marchdown 13y ago

I would love to see your take on this dependency graph, no matter how subjective and incomplete it turns out to be.

ivan_ah 13y ago
{arithmetic} | {vecops} = {Vector operations} {dotproduct} {arithmetic} | {RREF} = {Reduced row echelon form} {arithmetic} | {patience} | | {dotproduct} | | | {matrices} {Mops} = {Matrix operations} {Meqns} = {Matrix equations} {Mmult} = {Matrix multiplication} {Mdet} = {Determinants} {Minv} = {Matrix inverse} {y=mx+b} | {linear} = {Linearity} {plane} = {Lines and planes} {dotproduct} | {proj} = {Projections} {basis} = {Coordinate projections} {vectors} | {VS1} = {Vector spaces} {VS2} = {Abstract vector spaces} // knowledge buzz moment | | {dotproduct} | | {IPS} = {Abstract inner product spaces} {y=mx} | {Mmult} | | {function f:R->R} | | | {linT} = {Linear transformations} {VSRREF} = {Vector space techniques for matrices} {TasM} = {Finding matrix representations} {basis2} = {Change of basis for matrices} {Mmult} | {linT} | | {eigV} = {Eigenvalues and eigenvectors} {Mdecomp}= {Matrix decompositions} | {specM} = {Special types of matrices} {VS1} |{plane} ||{eigV} |||{linT} |||||{Meqns} ||||||{Mmult} ||||||||{Mdet} ||||||||| {invMthm} = {Invertible matrix theorem} {proj} | {GSorth} = {Gram--Schmidt orthogonalization} {RREF} | {LP} = {Linear programming} {Mmult} |{Meqn} ||{RREF} ||| {solve} = {Solving systems of equations} {LSaprx} = {Least squares approximate solution} {ML} = {Machine learning} {LAoverZ_q}= {Linear algebra over a finite field} ||| ||{ECCs} = {Error correcting codes} |{crypto} = {Cryptography} {netcode}= {Network coding} {trig} |{taylor} ||{AVS} |||{IPS} ||||| {fourier}= {Fourier series} {fourier} | {prob} = {Probability density} | | {LAoverC}= {Linear algebra with complex numbers} | | | {quantum}= {Quantum mechanics} UPDATE: Okay so if anyone is interested, I just put up a pre-release of the book https://gum.co/noBSLA https://gum.co/noBSLA