10 ms·
Algebra, Topology, Differential Calculus, and Optimization Theory for CS and ML [pdf]
- anarrayofatoms 7y agoWhat happened to the introduction?
- adamnemecek 7y agoI'm guessing they'll write it last. The book is still being written.
- hos234 7y agoMan...Math textbook/website/vid overload is making me weep for some reason. Most of this stuff hasn't changed in 200 years. Wtf do we need so many books for? Total failure of Math Pedagogy imho.
- george_ciobanu 7y agoCheck out Wolfram Alpha
- Myrmornis 7y agoI think this is a fair topic to bring up. Between commercial textbooks and freely-distributed PDFs, there is a huge amount of expositional material for any given topic in sub-research level mathematics and one does wonder whether there could be a better method of clustering and ranking the material. (Clustering is probably very doable.)
- pizza 7y agoNearly 2000 pages of dense material, mein Gott..
- jahbrewski 7y agoHonest question: how do you tackle 2k pages of material like this?
- rectangletangle 7y agoBinary search.
- shademaan 7y agoUse it as a reference
- george_ciobanu 7y agoI went to high school in Romania and first two years of college as well. We did tons of theory on about 2/3 (if not 3/4) of the material here (I also covered some of it when I continued my college in Montreal, Canada). But that took about 6 years of daily (maybe there was a day in the week when we didn't do math) study. And I'm sure we didn't cover it in the same depth as they do here (for most things we did a lot of proofs and theory). So it's possible but it takes a lot of time. In addition, it was usually motivated by the next step: you need to learn how to compute the inverse matrix in order to further process it with more complex algorithms which you actually use in labs or some sort of application (even if it's made up). The interesting thing is that, for a Math student (or a college professor) this is the FOUNDATION of math you need in order to study advanced stuff. For some of the recent math proofs (think famous ones) you'd need to study a whole year (after you know most of this stuff) just to learn the theory foundations for that proof. I'm of course exaggerating a bit - most Math majors would probably go through most of the stuff in this PDF but not necessarily all (machine learning stuff for example, and possibly numerical analysis stuff) of it. And if they do they usually forget* most of it, except the areas they use in their daily research (assuming they got for an advanced degree). And when you use some of this stuff every day it becomes second nature after a certain point. My experience is mostly based on the Romanian educational system which is typically more hardcore science (Math and Physics) even for Computer Engineering degrees. *Sometimes you don't forget stuff, you just don't think about it until you see it again (concrete example for me: Schur decomposition) and then partial memories of it come back.
- adamnemecek 7y agoThere's no way you covered 2/3 of this in hs.
- shademaan 7y agoJean Gallier (the author, and a CS prof at Penn) is quite the character. Also a very nice guy who cares a lot of about teaching. W Wish I'd read (some of) this book before I started my PhD!
- abhisuri97 7y agoPrevious discussion: https://news.ycombinator.com/item?id=20570025 https://news.ycombinator.com/item?id=20570025 Tl;dr: * This is dense * This is hard to understand if you don't have an education in mathematics * This is not really meant to be an "introduction to X" book unless you have the background knowledge * Gallier (who imo is an awesome professor) really values rigor and not hiding anything behind hand-wavy statements. He is an incredibly serious mathematician + computer scientist (https://en.wikipedia.org/wiki/Jean_Gallier https://en.wikipedia.org/wiki/Jean_Gallier). Accordingly, this book (work in progress) tries to be incredibly detailed on the topics it chooses to cover (and may miss out on some areas of CS + ML solely bc it chooses detail over comprehensiveness).
- chombier 7y agoA few years back I read Gallier's notes on matrix Lie groups, this was definitely the most accessible (yet complete) introduction to the subject I found.
- wenc 7y agoThe title is interesting because it purports to cover the areas of math relevant to CS and ML. So I took at a look at the topics covered, and my impression is that they were chosen by a generalist mathematician rather a specialist in each of the areas -- in optimization for instance, some of the topics would not be developed or weighted in quite in this fashion in a state-of-the-art course. Other subtopics serve purely theoretical interests -- either too theoretical or lack sufficient depth to be useful to the practitioner or the earnest researcher. I fear this may mislead some into thinking that the topics proposed are the primary ones in each area of study -- I would be wary of using this a trunk of a semantic tree to build knowledge on. There seems to a lack of mathematical unification in the work except perhaps these were a collection of topics that were of interest to the author(s). I want to say this could be a good handbook, but handbooks (e.g. engineering handbooks) are usually collections of pieces written by specialists in each subfield. I suspect that by attempting to cover such a large chunk of the mathematical landscape, some things may have been lost in the treatment (except in topics that the authors are specialists in). Still, it is a good effort and I congratulate the authors on the ambitious attempt but would advise readers to consult texts written by experts in each subfield to get a sense of what topics are important and what are not.
- ur-whale 7y ago200 pages of theory to build up to ... SVM's?? TBBH, that's kind of disappointing. I was hoping to see some sort of insight around deep learning stuff.
- pmorais 7y agoI don’t think this kind of insight even exists into deep learning stuff (yet). One of the main reasons people even care about SVMs is that they have nice analytical properties
- calf 7y agoWhere's Chapter 1? And wouldn't it be great if huge textbooks like these had a detailed dependency graph, so you could easily build up through a sequence of topic modules to reach something that interests you.
- siddboots 7y agoA while ago I did this for a few analysis text books that I was working through: Baby Rudin, Big Rudin, Calculus on Manifolds, and a couple of others. As I was working through the books I would capture every numbered definition or theorem in a table, along with all of the explicit cross-references or implicit dependencies. Then I wrote a little D3 app to visualise the graph, with a few customisable views. E.g what is the dependency structure of the chapters? What is the full list of dependencies for a given theorem? The result is actually pretty interesting, and the different levels of rigor that different authors choose really comes through. I would love to have a few weeks to sink into this project properly and make a public tool out of it.
- graycat 7y agoWhen I was writing my undergraduate honors paper on group representations, I tried something like your dependency structure! Result: Mostly all I got was just a simple sequence with nothing interesting or surprising. For that nearly 2000 page PDF, actually I know enough about the contents to suspect that again a dependency structure graph, especially for anyone with a background in Baby Rudin (Principles of Mathematical Analysis), Big Rudin (Real and Complex Analysis or maybe Functional Analysis), surprisingly, would not look very interesting. From just going through the table of contents, there are a lot of interesting and uncommon topics in those ~2000 pages, but the book looks more like a small encyclopedia or reference than an integrated whole. And for something that long and in places with some rare topics, there is, surprisingly, especially for the goal of ML (machine learning), comparatively little on probability, stochastic processes, and mathematical statistics. But in those many pages, there are many topics possibly valuable and not easy to find elsewhere, so keep a copy of the PDF and use it as a reference when appropriate. Here is a short remark for readers without much background: Don't be afraid to dig in nearly anywhere in the ~2000 pages; maybe you will have to backtrack a little but in general can start nearly anywhere and don't have to read sequentially chapter 1, 2, .... Warning: Some topics are covered relatively superficially while others have some really rare details. Broad, Curious Point: The ~2000 pages may be this and that, but it is relatively close to the applied mathematics of operations research and numerical analysis, closer to those fields than what is usually regarded as computer science until computer science wants to absorb the math of operations research and numerical analysis. For more, some of the numerical analysis was, as I recall, mostly of interest for numerical solutions of partial differential equations. That field is also to be subsumed by computer science?
- deleted 7y ago[deleted]
- xmly 7y agoIt is a great handbook! Thanks a lot.
- redsymbol 7y agoWow, this is epic! Nearly 2,000 pages, that's amazing. Weird as it sounds, some of the happiest moments of my life come from deeply studying mathematics like this. When I finally cash out of the entrepreneur game, I'm looking forward to spending several decadent hours each day studying something like this again. Great fun.
- cheez 7y agoYou don't need to cash out. Just build a life where you can take sabbaticals often. That's what I've done. It let me raise both my kids, one to college age, one almost there, on my own.
- Myrmornis 7y agoShould there exist better methods for deciding where one should allocate one's available time within the vast landscape of expositional mathematical material at undergrad and graduate level?
- armitron 7y agoSince it mentions CS I feel criticism is fair: I have a CS degree from CMU and this material is simply incomprehensible to me. It's not only written from a mathematical perspective but also aimed at mathematicians, and not computer scientists. Unfortunately, math textbooks that are both immediately useful and instantly accessible to computer scientists are few and far between. Why doesn't anyone strive to cover that gap? I find that the major point of contention with most math textbooks is their tendency to pile one definition after another without showing me how that definition is useful in practice. I have a limit in how many abstract mathematical structures and theorems I can absorb before my brain simply wipes them out since it deems them to be useless. For that reason, the best math textbooks for me have been grounded in practicality (and are not really math textbooks). I learned calculus from Spivak, abstract algebra from Antoine Joux and analysis from Sussman.
- knzhou 7y agoGiant, broad sets of notes like these are very satisfying for the writer -- not so much for the reader. I should know, since I have a similar 1500 page work on the foundations of physics, and nobody's ever found it comprehensible. [1] I think a lot of academics have these personal tomes lying around, though most haven't bothered to type them up. I see a lot of comments asking how one should read these things, but the answer is that one shouldn't. These documents are more like diaries than textbooks. They'll show you the scaffolding of somebody else's technical mind, not how to build your own. The best textbooks are always focused, and honed by teaching experience. 1: https://knzhou.github.io/#lectures https://knzhou.github.io/#lectures
- sn9 7y agoCal Newport has written about experimenting with this; he calls it the Textbook method[0][1]. [0] https://www.calnewport.com/blog/2012/08/10/you-know-what-you-write-the-textbook-method-for-ultra-learning/ https://www.calnewport.com/blog/2012/08/10/you-know-what-you... [1] https://www.calnewport.com/blog/2012/08/16/experiments-with-the-textbook-method/ https://www.calnewport.com/blog/2012/08/16/experiments-with-...
- deleted 7y ago[deleted]
- jolmg 7y agoI don't know about the whole book or how it compares to other textbooks, but as someone who didn't really know what exactly Groups, et al. were, I found that first page of Chapter 2 to be very clear and to-the-point. I'm sure not everyone learns the same way, but the way that first page didn't beat around the bush at explaining what Groups, Monoids, and Abelian Groups are seems to really appeal to me. Then again, maybe I just haven't experienced better material on that topic. Do you have any suggestions?
- knzhou 7y agoShort segments of a massive tome can be good! Honestly all decent expositions of groups are basically equally good, though different styles can still appeal more depending on the person. The problem is when people try to use the whole book for some goal -- whatever that goal is, it's never an efficient route.