12 ms·

Ok HN, what's the best book for learning this stuff? My criteria are: rigorous is good, but so is fun. Bonus points for anything a hacker would appreciate, such

by gruseom 15y ago

Ok HN, what's the best book for learning this stuff? My criteria are: rigorous is good, but so is fun. Bonus points for anything a hacker would appreciate, such as computational applications or exercises.

dxbydt 15y ago
How much time do you have ? You can easily spend several years and get multiple PhDs in linear algebra, but if you only have half an hour to devote to this, I strongly suggest Leduc's Cliff Notes. http://www.amazon.com/Linear-Algebra-Cliffs-Quick-Review/dp/0822053314 http://www.amazon.com/Linear-Algebra-Cliffs-Quick-Review/dp/... Amazing writing, very succint & hacker-friendly.
palebluedot 15y ago
I highly recommend the MIT 18.06 open course on linear algebra - the lectures are first rate. I've been going through this as a refresher, in prep for the Stanford machine learning class. Link to the course: http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/ http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-... Link to the YouTube videos of the lectures: http://www.youtube.com/watch?v=ZK3O402wf1c&feature=results_main&playnext=1&list=PLE7DDD91010BC51F8 http://www.youtube.com/watch?v=ZK3O402wf1c&feature=resul...
- achompas 15y ago
  The course is taught by Gilbert Strang, who has written an excellent introductory textbook on the subject: http://www.amazon.com/Introduction-Linear-Algebra-Fourth-Gilbert/dp/0980232716/ http://www.amazon.com/Introduction-Linear-Algebra-Fourth-Gil...
  - colanderman 15y ago
    Strang has another great book, Linear Algebra and its Applications: http://www.amazon.com/Linear-Algebra-Applications-Gilbert-Strang/dp/0030105676/ref=sr_1_2?ie=UTF8&qid=1315933580&sr=8-2 http://www.amazon.com/Linear-Algebra-Applications-Gilbert-St... Everything I know about linear algebra I learned from this book.
    - seanharnett 15y ago
      Seconded. I'm an applied math PhD student (optimization) and I use linear algebra constantly. This book taught me everything.
      - m0nastic 15y ago
        Thirded. We used an earlier edition at my school for Linear Algebra, and in spite of having a Professor who was retiring at the end of the class (complete with "I'm getting too old for this shit" demeanor), the book was approachable enough for us to get by.
    - cop359 15y ago
      A slightly more advance books which I love is: Matrix Analysis and Applied Linear Algebra by Carl Meyer. http://www.amazon.com/gp/product/0898714540 http://www.amazon.com/gp/product/0898714540 You can find a free PDF version online. I like it more then Strang because it's a lot more concise, covers some more advanced topics and unlike Strang everything is said very accurately. I think Strang's rather hand-wavy way of explaining things starts faltering when he talks about more advance topics. I would read Strang and listen to his lectures to get a good feel for Linear Algebra (to build up the intuition), and if you feel like you want more then pick up Meyer's book
lurker17 15y ago
johndcook's post is about numerical linear algebra. The recommendations given here so far are for intro/abstract linear algebra texts.
- gruseom 15y ago
  Correct. Though in my case I have to re-do Strang or the equivalent anyway. Been a long time.
  - mturmon 15y ago
    For the numerical stuff, it's hard to beat Golub and Van Loan. Texts that emphasize coordinate-free approaches, like Linear Algebra Done Right, don't really get at the numeric computations issue.
jparise 15y ago
Khan Academy has a complete Linear Algebra section: http://www.khanacademy.org/#linear-algebra http://www.khanacademy.org/#linear-algebra
brent 15y ago
Numerical Linear Algebra by Trefethen and Bao is well regarded.
tokenadult 15y ago
As the focus of the submitted article is on numerical methods, I'd recommend Hubbard's textbook http://matrixeditions.com/UnifiedApproach4th.html http://matrixeditions.com/UnifiedApproach4th.html which is theoretical and linked to other subjects of higher math, but by no means slights practical problems or numerical methods.
nokcha 15y ago
I've read and would recommend Linear Algebra Done Right by Sheldon Axler. http://www.amazon.com/Linear-Algebra-Right-Sheldon-Axler/dp/0387982582/ http://www.amazon.com/Linear-Algebra-Right-Sheldon-Axler/dp/... For numerical aspects of linear algebra (and other subjects), I've found Numerical Recipes to be quite helpful.
- phren0logy 15y ago
  I've heard good things about this one, as well as _Linear Algebra for Everyone_. http://amzn.com/8847018382 http://amzn.com/8847018382 From the book's intro: >But, suppose we asked a professional mathematician to step back a bit from his habitual way of speaking and write in a more linear fashion? And suppose we even asked more, for example, that he make his writing lively? ... The purpose of this book is to furnish the reader with the first mathemat- ical tools needed to understand one of the pillars of modern mathematics, i.e. linear algebra. The text has been written by a mathematician who has tried to step out of his usual character in order to speak to a larger public. He has also taken up the challenge of trying to make accessible to everyone the first ideas and the first techniques of a body of knowledge that is fundamental to all of science and technology.
- Silhouette 15y ago
  While Numerical Recipes has certainly earned its place in the history of the genre, it is only fair to point out that it is very out-dated by now, and that the example code is neither particularly well-written (from a software development point of view) nor freely usable in your own projects. As others have suggested, something like Golub and Van Loan is a better choice. The technical notes to go with the LAPACK library, and similar documents from those developing related software, are also likely to be of interest to anyone doing this stuff seriously.
muuh-gnu 15y ago
After the LA intro courses, see Strangs "Computational Science and Engineering": * http://www.youtube.com/view_play_list?p=F706B428FB7BD52C http://www.youtube.com/view_play_list?p=F706B428FB7BD52C * http://www.youtube.com/view_play_list?gl=DE&hl=de&p=3A13781649466805 http://www.youtube.com/view_play_list?gl=DE&hl=de&p=... based on the book of the same name. This is one of "name 5 books you would take with you on a desert island" kind of books, a beautiful work of art.
robobenjie 15y ago
The (free, open) stanford class ee263: Linear Dynamical Systems is really good. It concentrated on how to take real problems and fit them into a form where a computer can solve them in what appears to be complete magic. Plus Stephen Boyd is a very entertaining lecturer. This article reminds me a lot of his asides. Link to lectures and materials: http://see.stanford.edu/see/courseinfo.aspx?coll=17005383-19c6-49ed-9497-2ba8bfcfe5f6 http://see.stanford.edu/see/courseinfo.aspx?coll=17005383-19...
tptacek 15y ago
Ok, Daniel. Linear algebra study group. Let's make something happen. (I just bought the Strang book mentioned downthread).
- gruseom 15y ago
  I'm in. Which of the Strangen? And what's a suitable forum for working out the details and proceeding? A Google group would probably be my go-to default but I'm open to alternatives. Edit: Let's see if this works: http://news.ycombinator.com/edit?id=2993321 http://news.ycombinator.com/edit?id=2993321.
  - tptacek 15y ago
    I bought _Linear Algebra and its Applications_, but as I'm fond of reminding people on HN (come work for us!) my book budget is unbounded, so I'm happy to buy a different one.
pkrumins 15y ago
I am reviewing all MIT Linear Algebra lectures on my blog, including lecture notes and my comments. Start here: http://www.catonmat.net/blog/mit-linear-algebra-part-one/ http://www.catonmat.net/blog/mit-linear-algebra-part-one/
thisrod 15y ago
Wow. No one has mentioned the gospel according to Golub: http://www.amazon.com/Computations-Hopkins-Studies-Mathematical-Sciences/dp/0801854148/ref=sr_1_1?ie=UTF8&qid=1315956173&sr=8-1 http://www.amazon.com/Computations-Hopkins-Studies-Mathemati... This is the "how we wrote Lapack" book.
- alxv 15y ago
  Yes, Golub/van Loan is the bible. But Walkins's "Fundamentals of Matrix Computations" (http://www.amazon.com/o/asin/0470528338/ http://www.amazon.com/o/asin/0470528338/) is much more accessible to self-learners. For those who are just starting on this subject, I highly recommend that you rework the proofs for the bounds between the various matrix norms [1]. These bounds are the building blocks for most interesting analyses in matrix computations. And understanding these analyses are essential if you want to know which algorithm will work best for your problems. [1]: http://en.wikipedia.org/wiki/Matrix_norm#Examples_of_norm_equivalence http://en.wikipedia.org/wiki/Matrix_norm#Examples_of_norm_eq...
brendano 15y ago
I put together some free online resources for linear algebra: http://brenocon.com/blog/2011/03/poor-mans-linear-algebra-textbook/ http://brenocon.com/blog/2011/03/poor-mans-linear-algebra-te... Everyone I talked to then recommended the Strang textbook, and also Stanford EE263, both already pointed out by other commenters here.

by gruseom 15y ago

dxbydt 15y ago

How much time do you have ? You can easily spend several years and get multiple PhDs in linear algebra, but if you only have half an hour to devote to this, I strongly suggest Leduc's Cliff Notes. http://www.amazon.com/Linear-Algebra-Cliffs-Quick-Review/dp/0822053314 http://www.amazon.com/Linear-Algebra-Cliffs-Quick-Review/dp/... Amazing writing, very succint & hacker-friendly.

palebluedot 15y ago

I highly recommend the MIT 18.06 open course on linear algebra - the lectures are first rate. I've been going through this as a refresher, in prep for the Stanford machine learning class. Link to the course: http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/ http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-... Link to the YouTube videos of the lectures: http://www.youtube.com/watch?v=ZK3O402wf1c&feature=results_main&playnext=1&list=PLE7DDD91010BC51F8 http://www.youtube.com/watch?v=ZK3O402wf1c&feature=resul...

achompas 15y ago
The course is taught by Gilbert Strang, who has written an excellent introductory textbook on the subject: http://www.amazon.com/Introduction-Linear-Algebra-Fourth-Gilbert/dp/0980232716/ http://www.amazon.com/Introduction-Linear-Algebra-Fourth-Gil...
- colanderman 15y ago
  Strang has another great book, Linear Algebra and its Applications: http://www.amazon.com/Linear-Algebra-Applications-Gilbert-Strang/dp/0030105676/ref=sr_1_2?ie=UTF8&qid=1315933580&sr=8-2 http://www.amazon.com/Linear-Algebra-Applications-Gilbert-St... Everything I know about linear algebra I learned from this book.
  - seanharnett 15y ago
    Seconded. I'm an applied math PhD student (optimization) and I use linear algebra constantly. This book taught me everything.
    - m0nastic 15y ago
      Thirded. We used an earlier edition at my school for Linear Algebra, and in spite of having a Professor who was retiring at the end of the class (complete with "I'm getting too old for this shit" demeanor), the book was approachable enough for us to get by.
  - cop359 15y ago
    A slightly more advance books which I love is: Matrix Analysis and Applied Linear Algebra by Carl Meyer. http://www.amazon.com/gp/product/0898714540 http://www.amazon.com/gp/product/0898714540 You can find a free PDF version online. I like it more then Strang because it's a lot more concise, covers some more advanced topics and unlike Strang everything is said very accurately. I think Strang's rather hand-wavy way of explaining things starts faltering when he talks about more advance topics. I would read Strang and listen to his lectures to get a good feel for Linear Algebra (to build up the intuition), and if you feel like you want more then pick up Meyer's book

lurker17 15y ago

johndcook's post is about numerical linear algebra. The recommendations given here so far are for intro/abstract linear algebra texts.

gruseom 15y ago
Correct. Though in my case I have to re-do Strang or the equivalent anyway. Been a long time.
- mturmon 15y ago
  For the numerical stuff, it's hard to beat Golub and Van Loan. Texts that emphasize coordinate-free approaches, like Linear Algebra Done Right, don't really get at the numeric computations issue.

jparise 15y ago

Khan Academy has a complete Linear Algebra section: http://www.khanacademy.org/#linear-algebra http://www.khanacademy.org/#linear-algebra

brent 15y ago

Numerical Linear Algebra by Trefethen and Bao is well regarded.

tokenadult 15y ago

As the focus of the submitted article is on numerical methods, I'd recommend Hubbard's textbook http://matrixeditions.com/UnifiedApproach4th.html http://matrixeditions.com/UnifiedApproach4th.html which is theoretical and linked to other subjects of higher math, but by no means slights practical problems or numerical methods.

nokcha 15y ago

I've read and would recommend Linear Algebra Done Right by Sheldon Axler. http://www.amazon.com/Linear-Algebra-Right-Sheldon-Axler/dp/0387982582/ http://www.amazon.com/Linear-Algebra-Right-Sheldon-Axler/dp/... For numerical aspects of linear algebra (and other subjects), I've found Numerical Recipes to be quite helpful.

phren0logy 15y ago
I've heard good things about this one, as well as _Linear Algebra for Everyone_. http://amzn.com/8847018382 http://amzn.com/8847018382 From the book's intro: >But, suppose we asked a professional mathematician to step back a bit from his habitual way of speaking and write in a more linear fashion? And suppose we even asked more, for example, that he make his writing lively? ... The purpose of this book is to furnish the reader with the first mathemat- ical tools needed to understand one of the pillars of modern mathematics, i.e. linear algebra. The text has been written by a mathematician who has tried to step out of his usual character in order to speak to a larger public. He has also taken up the challenge of trying to make accessible to everyone the first ideas and the first techniques of a body of knowledge that is fundamental to all of science and technology.
Silhouette 15y ago
While Numerical Recipes has certainly earned its place in the history of the genre, it is only fair to point out that it is very out-dated by now, and that the example code is neither particularly well-written (from a software development point of view) nor freely usable in your own projects. As others have suggested, something like Golub and Van Loan is a better choice. The technical notes to go with the LAPACK library, and similar documents from those developing related software, are also likely to be of interest to anyone doing this stuff seriously.

muuh-gnu 15y ago

After the LA intro courses, see Strangs "Computational Science and Engineering": * http://www.youtube.com/view_play_list?p=F706B428FB7BD52C http://www.youtube.com/view_play_list?p=F706B428FB7BD52C * http://www.youtube.com/view_play_list?gl=DE&hl=de&p=3A13781649466805 http://www.youtube.com/view_play_list?gl=DE&hl=de&p=... based on the book of the same name. This is one of "name 5 books you would take with you on a desert island" kind of books, a beautiful work of art.

robobenjie 15y ago

The (free, open) stanford class ee263: Linear Dynamical Systems is really good. It concentrated on how to take real problems and fit them into a form where a computer can solve them in what appears to be complete magic. Plus Stephen Boyd is a very entertaining lecturer. This article reminds me a lot of his asides. Link to lectures and materials: http://see.stanford.edu/see/courseinfo.aspx?coll=17005383-19c6-49ed-9497-2ba8bfcfe5f6 http://see.stanford.edu/see/courseinfo.aspx?coll=17005383-19...

tptacek 15y ago

Ok, Daniel. Linear algebra study group. Let's make something happen. (I just bought the Strang book mentioned downthread).

gruseom 15y ago
I'm in. Which of the Strangen? And what's a suitable forum for working out the details and proceeding? A Google group would probably be my go-to default but I'm open to alternatives. Edit: Let's see if this works: http://news.ycombinator.com/edit?id=2993321 http://news.ycombinator.com/edit?id=2993321.
- tptacek 15y ago
  I bought _Linear Algebra and its Applications_, but as I'm fond of reminding people on HN (come work for us!) my book budget is unbounded, so I'm happy to buy a different one.

pkrumins 15y ago

I am reviewing all MIT Linear Algebra lectures on my blog, including lecture notes and my comments. Start here: http://www.catonmat.net/blog/mit-linear-algebra-part-one/ http://www.catonmat.net/blog/mit-linear-algebra-part-one/

thisrod 15y ago

Wow. No one has mentioned the gospel according to Golub: http://www.amazon.com/Computations-Hopkins-Studies-Mathematical-Sciences/dp/0801854148/ref=sr_1_1?ie=UTF8&qid=1315956173&sr=8-1 http://www.amazon.com/Computations-Hopkins-Studies-Mathemati... This is the "how we wrote Lapack" book.

alxv 15y ago
Yes, Golub/van Loan is the bible. But Walkins's "Fundamentals of Matrix Computations" (http://www.amazon.com/o/asin/0470528338/ http://www.amazon.com/o/asin/0470528338/) is much more accessible to self-learners. For those who are just starting on this subject, I highly recommend that you rework the proofs for the bounds between the various matrix norms [1]. These bounds are the building blocks for most interesting analyses in matrix computations. And understanding these analyses are essential if you want to know which algorithm will work best for your problems. [1]: http://en.wikipedia.org/wiki/Matrix_norm#Examples_of_norm_equivalence http://en.wikipedia.org/wiki/Matrix_norm#Examples_of_norm_eq...

brendano 15y ago

I put together some free online resources for linear algebra: http://brenocon.com/blog/2011/03/poor-mans-linear-algebra-textbook/ http://brenocon.com/blog/2011/03/poor-mans-linear-algebra-te... Everyone I talked to then recommended the Strang textbook, and also Stanford EE263, both already pointed out by other commenters here.