9 ms·
Fundamentals of Linear Algebra and Optimization [pdf]
- alexcnwy 9y agoThis is great! There's so much hype about machine learning but so few people seem to appreciate the fundamental importance of linear algebra (especially in ML). This fast.ai course is also great: http://www.fast.ai/2017/07/17/num-lin-alg/ http://www.fast.ai/2017/07/17/num-lin-alg/
- zengid 9y agoCool new course! I'm thinking about taking their Practical Deep Learing for Coders course at the moment: http://course.fast.ai/ http://course.fast.ai/
- edanm 9y agoIf you're at all interested in actually doing deep learning, then do the course! It's by far my favorite resource for actually learning to do deep learning. Of course it won't give you the math background, but you can abd should pick it up after.
- nafizh 9y agoWait till January when they will release version 2 of this course which teaches DL with pytorch. I was one of the international fellows for that course, and it is so good you would want to do the updated one.
- deleted 9y ago[deleted]
- glbrew 9y agoIs this going to be a textbook? I can't find it in print, but it looks like a great text.
- mindcrime 9y agoGood question. I'm honestly not sure. I just stumbled across it while looking for linalg resources. It appears to be legit, since the author is with UPenn and it's on a upenn.edu site. But I don't see an explicit license notification or anything, so not sure if you're even allowed to print it out yourself or not.
- mpowell94 9y agoWas in his class at Penn, he gives all this stuff away with no issue. Didnt have to pay anything for course materials and its all open to the public.
- da_chicken 9y agoIt's probably the course book that the professor or department wrote for the specific course because they couldn't find a textbook that covered the material they wanted in the way they wanted. A lot of courses at U of M had them in the 90s; you'd buy the course book from a local printer's shop as a tape-bound, soft-cover book. They were very nice because they covered exactly what you needed using the same language as the professor. It was like having notes you could read before the lectures.
- jacobolus 9y agoFolks looking for introductory books in this area may also enjoy this new in-progress book: https://web.stanford.edu/~boyd/vmls/ https://web.stanford.edu/~boyd/vmls/ As a follow-up I would recommend Trefethen & Bau, Numerical Linear Algebra.
- deleted 9y ago[deleted]
- JustFinishedBSG 9y agoBoyd and Vandenberghe will have a monopoly in anything optimization related if they keep on...
- dragandj 9y agoIf you're into Clojure, there is a fast linear algebra library Neanderthal (I'm the author) that covers both GPU and CPU, is easy to use, and comes with lots of tutorials that cover a linear algebra textbook chapters, as well as more advanced numerical computing uses (solvers, factorizations, etc.): The list of guides is here: http://neanderthal.uncomplicate.org/articles/guides.html http://neanderthal.uncomplicate.org/articles/guides.html
- avalonalex 9y agoI took Prof. Gallier (the author of this document)'s class when I was a graduate student at Penn. http://www.cis.upenn.edu/~jean/home.html http://www.cis.upenn.edu/~jean/home.html It is one of the class I enjoyed the most when I was there. Some of his other books are quite enjoyable as well. Dr. Gallier's PhD work was focused on logic, but his current research has a lot to do with computer graphics and computational geometry. He spent a long time study graduate-level algebraic geometry and algebraic topology while having the duty to be a professor at computer science department. His stories is always an inspiration for me to learn more.
- hvs 9y agoI may be a troglodyte, but I was bored senseless in my linear algebra, calculus, discrete mathematics, statistics, etc classes in college. Then they tag all the interesting stuff like AI (we didn't call it Machine Learning back in the 90s) at the end of your major where you actually use it. I had to go back and relearn it all because I didn't pay attention the first time. Note to mathematicians (or at least math professors): lead with the interesting stuff before the theory. It'll make learning easier and more fun.
- Eridrus 9y agoI'm basically in the same boat, except I thought our AI class was basically just A* and other forms of search and didn't go back to learning any of it until after university!
- closeparen 9y agoSame: I never gave a crap about algebra, or calculus, until AP physics. When I learned that the point (and the origin story!) of all this is to model the behavior of the universe I suddenly wished I had been paying attention for years prior. Seconding your wish that curricula would lead with motivations and then drill mechanics rather than drilling mechanics for 11 years and finally giving you the motivation in year 12.
- colmvp 9y agoIt reminds me a bit of the motivation behind fast.ai, that you should get your hands dirty with the things you care about instead of learning the tough things until you're ready. I found it easier to learn things like stats, linear algebra, and calculus once I had a personal application to it. Even something that I love in Reinforcement Learning, I would sometimes get sleepy reading pages and pages from the Sutton and Barto book, but as soon as I worked on a coding exercise, I could spend hours actively participate with trying to solve the problem.
- wolfgke 9y ago> lead with the interesting stuff before the theory. It'll make learning easier and more fun. As a mathematician I say: What is considered as interesting is different for each person. I, personally, for example deeply love this really abstract stuff (but of course I am aware that other people have different preferences). So I would say even finding "interesting stuff" that many students in the lecture hall might be interested in is really, really hard. Another important argument against your idea is: To be even able to formulate the ideas from AI, one first have to learn and understand the words of the language in which one will formulate this. And these words are like "vector space", "linear map", "tensor product" etc. and understanding their meaning means knowing theorems about them. Starting with advanced topics, such as your AI example, is like giving beginner language learners a really advanced text in the foreign language that they just begin to learn. In other words: A reallz dubious idea. Considerung your post I can only ask you why you did not have a talk with the course advisor of your faculty. He would immediately have told you why these lectures are important for the things that you are actually interested in.
- SonOfLilit 9y agoMy test for linear algebra books is how they first present matrices and matrix multiplication. If they define a matrix as an NxM table of numbers with a multiplication operation defined as this complicated formula with a couple of nested sigmas, and then much later a lemma is mentioned that says every linear transformation can be represented as a matrix and then the composition of two transforms is the matrix multiplication of their matrix forms, I throw the book away in disgust. I throw most books away in disgust. This is the second book I've seen that does it right, but unlike the other one [1] this one wraps the very correct presentation of matrices in so much technical language and such a boring cover story that I almost threw it away in disgust anyway. My greatest wish in STEM education is that we teach linear algebra better. It's such low hanging fruit and could change so much! [1] which I saw linked in HN a year ago and don't remember the name of, but it was something like "linear algebra taught the correct way" and was apparently well known in the States so ask your friends
- jpau 9y agoMight [1] be "Linear Algebra Done Right" by Axler? http://www.springer.com/gp/book/9783319110790 http://www.springer.com/gp/book/9783319110790
- andars 9y agoThe Friedberg, Insel, Spence book [1] passes your test. Linear transformations first, then matrices as a representation of transformations with respect to a specified basis. Depending on your tastes, you may find it too dry, but imo it is a clean presentation. [1]: https://www.amazon.com/Linear-Algebra-4th-Stephen-Friedberg/dp/0130084514 https://www.amazon.com/Linear-Algebra-4th-Stephen-Friedberg/...
- abecedarius 9y agoAnother text that starts with linear transformations is Apostol’s Calculus. It’s somewhat more concrete than Axler, IIRC — e.g. it doesn’t try to avoid determinants, but starts with properties of volume-scaling and then develops the determinant formula out of those. We used this text for freshman/sophomore math back in the 80s, so my memory is fuzzy; but I liked it a lot.
- 9y ago
- macawfish 9y agoI'd recommend watching these videos for some motivation: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x... And remember, linear algebra has broader application than just geometry!
- catnaroek 9y agoDid fields other than R and C eat the authors' children or something?
- jacobolus 9y agoOften linear algebra tools are applied to Hilbert spaces or Banach spaces, where fields other than R or C don't make much sense. Looking at page 25 of TFA: > In Definition 1.2, the field R may be replaced by the field of complex numbers C, in which case we have a complex vector space. It is even possible to replace R by the field of rational numbers Q or by any other field K (for example Z/pZ, where p is a prime number), in which case we have a K-vector space (in (V3), ∗ denotes multiplication in the field K). In most cases, the field K will be the field R of reals.
- catnaroek 9y agoYeah, lots of courses pay lip service to the existence of fields of characteristic p during the initial few weeks, then proceed to ignore them completely.
- jacobolus 9y agoHow relevant do you think finite fields are for solving applied optimization problems? What proportion of a course do you think should be devoted to them?
- catnaroek 9y agoEvery ordered field has characteristic 0, so I don't think finite fields are likely to be terribly useful for solving optimization problems. But: (0) There is way more to linear algebra than linear programming. (1) A linear algebra course is not the right place for an extensive study of how to solve optimization problems anyway. That belongs in a real analysis course.
- jacobolus 9y ago
- Nydhal 9y agoYou had me at Bourbaki.