5 ms·
A First Course in Linear Algebra
- ColinWright 14y agoLinear Algebra is a basis for so much in Machine Learning, and this is a free, open textbook that takes the beginner through the basics. I've not read it myself - I'm not the intended audience - but I've seen several reviews from people whose opinions I trust, and it seems to be well respected.
- dbbolton 14y agoHonestly, I felt like linear algebra was the low point in my math career-- even behind axiomatic set theory. I still have nightmares about determining whether a set of vectors is linearly independent.
- shurane 14y agoMaybe you just had a bad approach? Linear algebra is loads of fun, methinks. And you need a good teacher or appreciation of the material for that. And it's something you slowly forget if you don't put it to use.
- impendia 14y agoLinear algebra is frequently taught very poorly. I think most university courses in the subject are kind of watered down and reduced to algorithms you can memorize -- which, ironically, makes them tough going for the strong students.
- wickedchicken 14y ago> I felt like linear algebra was the low point in my math career What do you consider to be the high point?
- hobbyist 14y agoKhan Academy's lectures on linear algebra is more than enough for a hacker to get the gory math details and then jump into machine learning.
- paperwork 14y agoAre there any linear algebra books which use machine learning as the motivating example? Or a book which teaches linear algebra and machine learning together?
- j2kun 14y agoSadly you need a hefty amount of it to get to machine learning that uses linear algebra in a nontrivial way.
- kylebrown 14y agoFunny, I just spent last night implementing a "principal compnents analysis" using numeric.js, for a character recognition project. PCA / SVD seems like a great motivating topic. btw, I couldn't use Singular Value Decomposition in numeric.js for PCA because the method, numeric.svd, uses the "thin" algorithm, and throws an error if there are more columns than rows. I calculate way more features (50-200+ columns) than I have training samples (rows, 10-30 written manually). without svd I had to use the "covariance method", which I guess can sometimes present approximation issues, but seems to be working well for me. The purpose of the PCA is dimensionality reduction (google "curse of dimensionality"). I had used Mahalanobis Distance as a p-value score to detect outliers (p-val < 0.05), and it worked well when there were only 6 features. Curse of dimensionality makes MD useless when there are 50 or 100 features, and PCA reduces them to 3-10 features which carry the most information with the rest approaching zero. So I do MD on the projected (reduced) features, and its working great. If I do it all over again I might try a "one-class SVM" (which, sadly, I had not heard of until only recently and very late in the project). SVM's are non-linear like most machine learning algos, but the linear PCA can still be used to complement other methods, eg to do pre-processing before feeding to a neural network.[1] For deeper linear methods, check out the PCA-related extensions like Fisher Linear Discriminants or Projection Pursuit. 1. "Many neural networks are susceptible to the Curse of Dimensionality though less so than the statistical techniques. The neural networks attempt to fit a surface over the data and there must be sufficient data density to discern the surface. Most neural networks automatically reduce the input features to focus on the key attributes. But nevertheless, they still benefit from feature selection or lower dimensionality data projections."
- j2kun 14y agoThis book spends three chapters on the mechanics of matrix operations before defining a vector space, and then it jumps into determinants. IMO this is the wrong way to understand linear algebra, and the typical path of a lot of introductory text books. A better text: Axler's "Linear Algebra Done Right"
- pvarangot 14y agoA million times this. Its not trivial to geometrically understand linear algebra without getting into topology if you go "determinants first". An algebraic understanding of linear algebra is not the most adequate if you are learning it to further understand topics such as machine learning or signal processing. PD: If you really do like or need the algebraic approach, Shilov's book is the way to go after this course... I think the first chapter is about determinants and he builds the rest of the book from there. It also includes a chapter on tensor algebra.
- impendia 14y agoDefinitely agreed. As a math professor, I think you can get by without defining vector spaces -- I think it is a little bit difficult to come up with examples, other than R^n or C^n, which seem motivated and interesting to the beginner. The truly important example (IMHO) is R^n without a choice of basis -- but I think this can only be well motivated after you've seen a lot of linear algebra, not before. Mechanics of matrix operations are not pleasant to teach, they make the subject seem like a bunch of contrived and confusing examples. Same for the "row echelon" stuff. Alright, you now have an algorithm for solving systems of equations... but presenting linear algebra as an algorithm sells it short. What is really important in my view is the geometry of the subject, which is already very interesting in two dimensions. Problem: Here is a linear transformation, given as a 2x2 matrix. Draw a picture which illustrates what this does to the plane. If you ask me, this is much more important than most of the crap that gets taught and tested in most courses in linear algebra.
- alecdibble 14y agoI completely agree. In college, my first set of Linear Algebra courses were taught by the Math department and were completely algebra based. Later on, taking the engineering LA courses where they stressed the geometry of all the operations really cemented the concepts and simplicity of LA into my mind.
- purephase 14y agoAnyone have any recommendations for linear algebra courses through Coursera or similar sites? It is a subject I've always been fascinated by but my self-directed learning endeavours have not yielded much usable knowledge.
- cottonseed 14y agoMIT OCW has 18.06 Linear Algebra including lectures by Strang.
- rrmm 14y agohttp://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-lectures/ http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-... I agree, it's very useful. He gives nice explanations of some of the decompositions and why they're useful.
- shriphani 14y agoRuslan Sharipov wrote a text on the subject : http://arxiv.org/pdf/math/0405323v1.pdf http://arxiv.org/pdf/math/0405323v1.pdf It is very good and explains all the proofs and definitions very well.
- mindcrime 14y agoOn a slightly related note, if anyone is looking for more free resources for learning maths of various sorts, check out: http://mathbooks.reddit.com http://mathbooks.reddit.com and/or http://people.math.gatech.edu/~cain/textbooks/onlinebooks.html http://people.math.gatech.edu/~cain/textbooks/onlinebooks.ht...
- iamalexalright 14y agoI'll post my favourite (sorted by topic - many of them free) http://hbpms.blogspot.com/ http://hbpms.blogspot.com/
- mindcrime 14y agoOooh, very cool resource. I'd been looking for something like that for a while actually. Something that gives a little bit of structure in terms of what sequence to do things in... that's cool.
- Miserlou57 14y agoSee how everyone has their own favorite teacher / explanation? Let's aggregate them all. http://www.reddit.com/r/newschool/comments/wmu5q/read_this_how_newschool_works/ http://www.reddit.com/r/newschool/comments/wmu5q/read_this_h...
- santigepigon 14y agoWhat impresses me is the accessibility of this textbook. Not only is it available online (via XML and MathML) but you can download the TeX source and multiple PDFs optimized for printing or on-screen viewing [1]. Bravo. [1] http://linear.ups.edu/download.html http://linear.ups.edu/download.html
- deleted 14y ago[deleted]
- tmh88j 14y agoThis is the best resource I've used for higher level math. I'm not sure why more people don't know about it because everything is VERY in depth with great examples and explanations. http://tutorial.math.lamar.edu/Classes/LinAlg/LinAlg.aspx http://tutorial.math.lamar.edu/Classes/LinAlg/LinAlg.aspx
- Watabou 14y agoI've used Paul's Online Notes for all my higher level math classes starting from Calculus 2. It has been more useful to me, even more than Khan's Academy. I like the little tricks and mnemonics that he teaches, especially in Linear Algebra. Example here: http://tutorial.math.lamar.edu/Classes/LinAlg/DeterminantReview.aspx http://tutorial.math.lamar.edu/Classes/LinAlg/DeterminantRev.... Much easier way of calculating determinants. Another resource I've used is Patrick JMT's excellent videos found at: http://patrickjmt.com http://patrickjmt.com. He really goes over problems slowly. Best math teacher I've ever had.
- wickedchicken 14y agoThank you for this. I had wanted to brush up my knowledge on vector spaces for a while and this looks like the kick I needed.