9 ms·
Linear Algebra for AI
- methodover 9y agoIs there a website that could interpret tbis... iPython notebook, I think it is? I'd love to read it on mobile.
- fsqcds 9y agoGitHub does it. Try to request desktop version.
- c8g 9y agoMobile friendly: https://nbviewer.jupyter.org/github/fastai/fastai/blob/master/tutorials/linalg_pytorch.ipynb https://nbviewer.jupyter.org/github/fastai/fastai/blob/maste... View from github URL: https://nbviewer.jupyter.org/ https://nbviewer.jupyter.org/
- alishan-l 9y agoThanks for sharing!
- backpropaganda 9y agoAll the linear algebra you need for AI: http://cs229.stanford.edu/section/cs229-linalg.pdf http://cs229.stanford.edu/section/cs229-linalg.pdf
- ivan_ah 9y agoMore linear algebra stuff to read with you weekend morning coffee: https://minireference.com/static/tutorials/linear_algebra_in_4_pages.pdf https://minireference.com/static/tutorials/linear_algebra_in... See also Section VI in this SymPy tutorial https://minireference.com/static/tutorials/sympy_tutorial.pdf https://minireference.com/static/tutorials/sympy_tutorial.pd... or LA topics covered with numpy http://ml-cheatsheet.readthedocs.io/en/latest/linear_algebra.html http://ml-cheatsheet.readthedocs.io/en/latest/linear_algebra...
- derekmcloughlin 9y agoI found that chapter 2 of the Deep Learning book by Ian Goodfellow, Yoshua Bengio and Aaron Courville is quite good. http://www.deeplearningbook.org/contents/linear_algebra.html http://www.deeplearningbook.org/contents/linear_algebra.html
- tw1010 9y agoRight now, sure. But give it five years and you'll no doubt also need at least a years worth of undergraduate (pure) math.
- dragandj 9y agoAll the linear algebra that is needed for AI is matrix multiplication and broadcasting? Am I the only one who is baffled by this?
- tw1010 9y agoI think they forgot to prepend "to convince the math-phobic you know what you're doing"
- jventura 9y agoIf AI == Neural Networks only, maybe matrix knowledge is enough, but when you start to include NLP, Expert systems, SEarch algorithms such as A*, Minimax, etc. in the AI category, you'll have to know more mathematics.. AI is much more than neural networks!
- dragandj 9y agoEven for NN alone, those few operations are not enough. Not only that, but to even understand what the operations that I need directly do, I need far more background knowledge of linear algebra.
- contravariant 9y agoWell, if you want to simplify even more then all you really need is optimisation and a sufficiently broad class of functions.
- Yuioup 9y agoThank you for this! I've been re-learning Linear Algebra since my question[1] was answered on an earlier HN thread[2] on fast.ai. This will definitely help. [1] https://news.ycombinator.com/item?id=14878199 https://news.ycombinator.com/item?id=14878199 [2] https://news.ycombinator.com/item?id=14877920 https://news.ycombinator.com/item?id=14877920
- hal9000xp 9y agoIs there any book which actually explains where matrix and its rules come from? Instead of throwing on you matrix multiplication rules in dogmatic way so you blindly follow them like mindless robot? I know there are lectures in YouTube from 3Blue1Brown: https://www.youtube.com/watch?v=kjBOesZCoqc&list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://www.youtube.com/watch?v=kjBOesZCoqc&list=PLZHQObOWTQ... So I want a book which covers linear algebra in the similar manner.
- foota 9y agoThere was a web book I found a while ago that built up some sort of motivation for linear algebra. Unfortunately I don't remember what it was or the title. Edit: found it, https://graphicallinearalgebra.net https://graphicallinearalgebra.net Ymmv. Matrix multiplication is defined the way it is imo because it has interesting properties that way. Not very satisfying though.
- dmit 9y agohttp://linear.axler.net/ http://linear.axler.net/
- hexrcs 9y agoI second this. Linear Algebra Done Right is an awesome book. It also comes with a helpful selection of exercises after each chapter with detailed answers available on the website, which is great for self-learners. If you are a student and your Uni has a Springer subscription, you might be able to get the PDF for free.
- jasode 9y ago"Linear Algebra Done Right" is a fine book but its enduring popularity leads people to recommend it as a universal default answer. The parent asked if there was a LA book that covered the material in the same style as 3Blue1Brown's videos. If that's the criteria, Sheldon Axler's book isn't the best book. One can compare a sample chapter to the youtube videos and realize they use different pedagogy: http://linear.axler.net/Eigenvalues.pdf http://linear.axler.net/Eigenvalues.pdf
- 9y ago
- aqsalose 9y ago"All the Linear Algebra You Need". The title is misleading. In general, I like an applications-first approach, but I have hard time liking a tutorial with the title like that and which proceeds to say things like >Neural networks consist of linear layers alternating with non-linear layers. without providing a proper definition nor intuition what is linear and what is not. Next step they are constructing NN layers of ReLU's and not telling what is a rectified linear unit and only barely hinting it is supposed to do ("non-linearity"). The article is not a worthless, it's a nice tutorial to building a NN classifier for MNIST, but don't expect full mastery of the mathematics relevant in understanding NNs after reading this tutorial. Chapter 2 of the book by Goodfellow et al. is a serviceable concise summary of the relevant concepts, but I don't think that chapter alone is a good primary learning material if you are not already familiar with the subject. For that I'd recommend reading a proper undergrad level textbook (and doing at least some of the problem sets [1]), one example: http://math.mit.edu/~gs/linearalgebra/ http://math.mit.edu/~gs/linearalgebra/ and then continue with e.g. the rest of the Goodfellow's book. [1] There's no royal road into learning mathematics. Or for that matter, learning anything.
- jph00 9y agoIt's the notes for a 40 min talk Rachel will give at the O'Reilly AI conference. It was originally meant to be a longer tutorial, so the scope had to be cut down significantly, whilst the title remained. There's lots of links in the notebook to additional resources with more background info. Having said that, there really isn't much more linear algebra you need to implement neural networks from scratch. You'll need convolutions of course, although that's not too different from what's shown here. For those interested in much more detail, Rachel has a full computational linear algebra course online http://www.fast.ai/2017/07/17/num-lin-alg/ http://www.fast.ai/2017/07/17/num-lin-alg/ . Most of that isn't needed for most deep learning, however.
- deleted 9y ago[deleted]
- jwdunne 9y agoHrm. I find a lot of stuff on linear algebra not so intuitive or motivating. It's a bit harder to get through. A big motivator is the sheer scale of application, along with calculus. Yeah it's a must for AI but really linear algebra underpins much much more and gives you awesome problem solving tools in general. As for intuition, a better approach may be to get 3 good books and cycle through them. Where one doesn't feel so intuitive at a given point, the other 2 might. They will explain things in a different way with different examples. Connecting the ideas across 3 sources will help it sink in. For a light intuitive introduction, give this a try: https://betterexplained.com/articles/linear-algebra-guide/ https://betterexplained.com/articles/linear-algebra-guide/ Maybe it's a little too light but the idea is important: voraciously grasp the ideas intuitively. If none are forthcoming, try to make one. Another Linear Algebra book, by Heffernan, was a 'third' book: http://joshua.smcvt.edu/linearalgebra/ http://joshua.smcvt.edu/linearalgebra/ It's free too. I think someone else has mentioned No Bull Linear Algebra. That's a cool book too. I had to put my learnings aside a few months ago but I made more progress this way than just following one book.
- auston 9y agoThis left me with a lot of questions, it could definitely use more explanations between code examples. I had to do a lot of "reading between the lines". I've been reading https://www.manning.com/books/grokking-deep-learning https://www.manning.com/books/grokking-deep-learning and have been liking it better than this tutorial.
- bra-ket 9y ago"Coding the Matrix" : https://www.amazon.com/Coding-Matrix-Algebra-Applications-Computer/dp/0615880991 https://www.amazon.com/Coding-Matrix-Algebra-Applications-Co...
- colmvp 9y agoI wonder why there's so much more emphasis on Linear Algebra over Calculus (I've seen a number of courses teaching LA to complement DL courses), given that without Calculus, it's hard to understand optimization and backpropagation. Rote memorization might help for copying the same network over and over but isn't enough when you have to customize it.
- chestervonwinch 9y agoHow do you know a stationary point is a local optimum? Eigenvalues of the Hessian matrix! Even though vector calculus is taught without assuming linear algebra, a lot of the material is coupled. Further, many of the "first steps" machine learning ideas are essentially linear algebra problems (e.g., linear regression, PCA, etc).
- BenGosub 9y agoI have yet to go through the tutorial. LA is one of the areas where I am trying to improve... Coding the Matrix course/book is well written for those that find a good/intuitive approach to learning.
- RayVR 9y agoFirst line of fastai imports is a big "Fuck you" to usability. Don't import *.
- 1024core 9y ago> The purpose of this notebook is to serve as an explanation of two crucial linear algebra operations used when coding neural networks: matrix multiplication and broadcasting. Um... no. While I like Rachel, and she's really smart and all that, this is a vaaast oversimplification.
- nnfy 9y agoI may be biased because I have a background in mathematics, but after working with tensorflow for about 6 months now, I dont think you really need to understand linear algebra or multivariate calculus to work with neural nets, unless you're trying to implement your own engine. Edit: I dont mean to discourage anyone from learning, as an understanding of the relatively simple mathematics behind NNs may afford the user an additional intuition about the behavior of neural nets, but it appears that one can treat neural nets almost like black boxes, given a suitable engine to work with like TF. As an aside, TF is a pretty magnificent library. It pretty much works out of the box, and in addition to python and CPP bindings, there appears to be an unnoficial port to c#, although I haven't tried it yet. I strongly recommend the tutorials at tensorflow for anyone interested in experimenting.
- trentmb 9y agoYeah, you don't need to be an automotive engineer to drive a car. Most 'work' in AI seems to be using python to shovel data into libraries anyways.