12 ms·

Math Basics for Computer Science and Machine Learning [pdf]

883 points by oldgun 7y ago

planetabhi 7y ago
This is a good book
tempodox 7y ago
This book looks great, I've been looking for a resource like that.
merlinsbrain 7y ago
I love that they have problems you can solve as well at the end of (almost) every chapter. This IS a lot of math (1,962 pages) and it’s missing a preface/introduction which would have been helpful to understand if I need to go linear or if a la carte is okay. At the moment I’d assume each major section is independent. Awesome find! Wonder how It’s used. (One of) the author(s) seems pretty prolific too - http://www.cis.upenn.edu/~jean/ http://www.cis.upenn.edu/~jean/
- Koshkin 7y ago
  > a la carte Yeah, I wish we had an online resource (other than Wikipedia) anyone could learn any sort of math from in a systematic way... Oh well.
  - MikeTheGreat 7y ago
    I'm wondering - is this a genuine request, or a snarky, implicit reference to an online resource for learning math somewhere? I'd love to know about the existing resource, if it exists. (The only thing that comes to mind is Wolfram Alpha, which didn't seem 'systematic' the last time I skimmed the main page)
    - house9-2 7y ago
      Maybe they were referring to Khan Academy? https://www.khanacademy.org/math https://www.khanacademy.org/math
      - pirocks 7y ago
        I think Khan academy for higher math/physics/chemistry /CS would be invaluable. Maybe it could be crowdsourced in some way?
        kebman 7y ago
        Well, there's always 3Blue1Brown: https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw Take a look at his course, Essence of Calculus: https://www.youtube.com/watch?v=WUvTyaaNkzM&list=PLZHQObOWTQDMsr9K-rj53DwVRMYO3t5Yr https://www.youtube.com/watch?v=WUvTyaaNkzM&list=PLZHQObOWTQ... I'm not fluent in math, but I find it fascinating to watch, for some reason. NancyPi is also great: https://www.youtube.com/channel/UCRGXV1QlxZ8aucmE45tRx8w https://www.youtube.com/channel/UCRGXV1QlxZ8aucmE45tRx8w Oh, and don't forget the Mathologer: https://www.youtube.com/channel/UC1_uAIS3r8Vu6JjXWvastJg https://www.youtube.com/channel/UC1_uAIS3r8Vu6JjXWvastJg What a great guy! While I'm at it, go watch Numberphile too: https://www.youtube.com/user/numberphile https://www.youtube.com/user/numberphile Is there anyone I forgot? :D
  - gonza 7y ago
    The way i'm trying to learn math right now is with https://ocw.mit.edu https://ocw.mit.edu, then you can look up into CS or Math programs to progress with other courses
  - bigred100 7y ago
    https://courses.maths.ox.ac.uk/overview/undergraduate#37105 https://courses.maths.ox.ac.uk/overview/undergraduate#37105 Oxfords course stuff provides some structure to the interested user
  - graycat 7y ago
    What would you like to know? Ask and here you might receive ....
- deleted 7y ago
  [deleted]
- piggybox 7y ago
  Every time I came across a new book on HN, I feel more needs to be done in my rest of life :)
Gene_Parmesan 7y ago
From the start of Chapter 2: "In the following four chapters, the basic algebraic structures (groups, rings, fields, vectorspaces) are reviewed, with a major emphasis on vector spaces. Basic notions of linear algebra such as vector spaces, subspaces, linear combinations, linear independence, [...], dual spaces,hyperplanes, transpose of a linear maps, are reviewed." If anyone needs to start even earlier than this, I've actually found "3D Math Basics for Graphics and Game Development" to be a good true intro for linear algebra-related stuff. I think this would probably hold even if your primary interest is something other than graphics/game dev. Some of the text in that book's intro is a little cringey with its reliance on kind of juvenile game references, but I didn't find that sort of writing continuing during the actual text. So just push past that stuff. I got a copy of it to act as a refresher before diving into Real-Time Collision Detection since it's been quite a long time since formal math for me (as in, high school, because I'm self-taught in CS). I've managed to make up a lot of ground by working hard and finding classes to audit online (Strang's linear alg course on OCW is a good one), but I have found that depressingly few math texts which claim to be "introductory" are actually truly introductory. This isn't a slight against the linked work, I absolutely love when profs make resources such as this freely available. "How to Prove It" and "Book of Proof" are also great intros to formal math, if less immediately practical.
- hx2a 7y ago
  > If anyone needs to start even earlier than this, I've actually found "3D Math Basics for Graphics and Game Development" to be a good true intro for linear algebra-related stuff. Did you mean to write "3D Math Primer for Graphics and Game Development" [1]? If you did, I agree 100%. I got a lot out of this book and was able to put it to good use for several projects. [1] https://www.amazon.com/Math-Primer-Graphics-Game-Development-ebook/dp/B008KZU548/ https://www.amazon.com/Math-Primer-Graphics-Game-Development...
- codesushi42 7y ago
  I would disagree about the gamedev book reference, unless you are referring to the real basics of linear algebra. The really important concepts for ML are least squares, eigenvalues and vectors, and SVD. Those concepts are not very relevant to game programming. Well, least squares can be solved with projection, which is relevant for converting between coordinate spaces. But game dev isn't going to give you that intuition.
melodrama 7y ago
Great book. I think it's a good time to mention a couple of nice books (related) 1. Elementary intro to math of machine learning [0]. Its style is a bit less austere than that of OP's. It also has a chapter on probability. It could possible serve as a great prequel to the book linked in the OP. 2. The book on probability related topics of general data science: high-dimensional geometry, random walks, Markov chains, random graphs, various related algorithms etc [1] 3. Support for people who'd like to read books like the one linked in the OP, but never seen any kind of higher math before [2]. This book has a cover that screams trashy book extremely skimpy on actual info (anyone who reads a lot of tech books knows what I am talking about), but surprisingly,it contains everything it says it does and in great detail. Not even actual math textbooks (say, Springer) are usually written with this much detail. Author likes to add bullet point style elaboration to almost every definition and theorem which is (almost) never the case with gazillions of books usually titled "Abstract Algebra", "Real Analysis", "Complex Analysis" etc. Some such books sometimes attach words like "friendly" to their title (say, "Friendly Measure Theory For Idiots") and still do not rise to the occasion. Worse yet, a ton (if not most) of these books are exact clones of each other with different author names attached. The linked book doesn't suffer from any of these problems. [0] Mathematics For Machine Learning by Deisentoth, Faisal, Ong https://mml-book.github.io/book/mml-book.pdf https://mml-book.github.io/book/mml-book.pdf [1] Foundations Of Data Science By Blum, Hopcroft, Kannan http://www.cs.cornell.edu/jeh/book%20no%20so;utions%20March%202019.pdf http://www.cs.cornell.edu/jeh/book%20no%20so;utions%20March%... 2] Pure Mathematics for Beginners: A Rigorous Introduction to Logic, Set Theory, Abstract Algebra, Number Theory, Real Analysis, Topology, Complex Analysis, and Linear Algebra by Steve Warner https://www.amazon.com/Pure-Mathematics-Beginners-Rigorous-Introduction/dp/0999811754 https://www.amazon.com/Pure-Mathematics-Beginners-Rigorous-I...
- iamcreasy 7y ago
  I'll check out the last one. Currently I am teaching myself real analysis.
- deleted 7y ago
  [deleted]
bigred100 7y ago
That’s... quite a lot of math
markus_zhang 7y ago
That's almost 2,000 pages of math...I don't know why and how, but somehow I forgot most of the Statistics knowledge I obtained as a graduate student (in Stat) 10 years ago. I remembered that I took an advanced course about Bayesian Inference, and one course about Multivariate Statistics (PCA, Factor analysis, these kind of things), and my project is about Bernstein Polynomial. That's it...
- xenihn 7y ago
  You forget complex things that you don't use regularly. Math, spoken languages, written languages, coding...of course you can re-learn it, and re-learning is faster than learning it for the first time. Based on speaking to my managers in the past, it seems like a year-long lapse is enough for you to lose an incredible amount of retained knowledge/skill. But it's not a permanent loss.
  - markus_zhang 7y ago
    Yeah agreed, sometimes reading a research paper from the DS team would actually ring a bell somewhere and I know where to look at. I'm re-learning Statistics from bottom up at the moment lol but this 2,000-page book really looks daunting. I'm pretty sure I didn't take any advanced optimization course back in university.
SKILNER 7y ago
Very first sentence of 2.1 is full of notation, symbols and terms that I, as a prospective student, might not understand. So many teachers seem incapable of stepping outside their sphere of knowledge and seeing what they know and others do not. And so much work went into this.
- lone_haxx0r 7y ago
  On the other hand, it's a perfect refresher for those of us who "know" this, but somehow forgot most of it.
- vonholstein 7y ago
  As someone in the same boat I've found this book to be very helpful. https://www.amazon.com/gp/product/1466230525 https://www.amazon.com/gp/product/1466230525 - Mathematical Notation: A Guide for Engineers and Scientists
  - alexbanks 7y ago
    I purchased this. I've been trying to brush up on CS fundamentals (it's been a long time since college), but I get stuck just on trying to understand what I'm being asked to learn. Thank you
- bradlys 7y ago
  Even as someone who did their undergrad in math, it's a bit tough to digest. I'd have to look up a lot of the terms again. It's been five years since I was in college. Shows how little I've used it all since I graduated. It definitely looks more like, "math basics" for x field. Kinda like "automotive basics" for Honda or Ford vehicles. Where it's presumed that you know a lot of automotive lingo to begin with and you just need to know what spark plugs go with what engine. And not, "what is a spark plug?"
- graycat 7y ago
  Here's what he is doing. He wants to start with the set of real numbers, intuitively the points on the line, usually denoted by R, maybe typed in some special font. Then he wants to define, say, addition of real numbers. So, given two real numbers, x and y, that might be equal, he wants to define x + y. So, here he wants to regard addition, that is, +, as an operation. Then, as is usual for defining operations, he wants an operation to be just a special case of a function. So, he wants to call + a function. So, + will be a function of two variables, say, x and y. With usual function notation we will have +(x,y) = x + y The set of all (x,y) is the domain of the function, and the set of all x + y is the range. So, that defines the function + except commonly in pure math we want to be explicit about the range and domain of the function. For function +, the range is just the set of all pairs (x,y) with x and y in R. That set is also the set theory Cartesian product of set R with itself and written R x R. So, the domain of + is R x R. The range is just R. Then to be explicit about the range and domain of function +, we can write +: R x R --> R which says that + is a function with range R x R and domain R. We learned how to add in, what, kindergarten? So, why make this so complicated? Well, he wants to regard the real numbers as just one example of lots of different algebraic systems, e.g., groups, fields, vector spaces, and much more, with lots of operations and, possibly, more that could be defined. E.g., later in his book he will want to add vectors and matrices, take an inner product of two vectors, and multiply two matrices. So, back to addition on the real numbers, he wants to regard that as just a special case of an operation on an algebraic system. IMHO there's not much benefit for making adding two real numbers look so complicated. Whatever he did in that chapter for defining addition on the reals, soon he is discussing matrix multiplication with no definition at all -- assuming the reader already understands that, that is defined and discussed many pages later in his book. So, in his notation +: R x R --> R and matrix multiplication, he is using material before he has defined it, even before he has motivated, explained, exemplified, indicated the value of, and defined it. In good math writing and in good technical writing more generally, that practice is, in non-technical language, a bummer. But from the table of contents, it appears that the book has quite a long list of possibly interesting narrow topics. And maybe for the routine material, his proofs and presentation are good -- maybe. I thought enough of the book to keep a copy of the PDF. It's there; if someday I want a discussion of some narrow topic, maybe I'll try his book! In mathematical writing, it used to be common for the word processing to be much more work than the mathematics! Now with TeX and LaTeX, and I'm assuming that the book used one of these two, the flood gates are open!
mjortberg521 7y ago
Great to see Prof. Gallier featured on here!
jaimex2 7y ago
I look at this and profoundly thank the people who make ml libraries for us the rest of us.
- vecter 7y ago
  The vast vast vast vast vast majority of this book (which is more of a reference and encyclopedia than an actual book for learning) is not required for implementing most ML libraries.
krosaen 7y ago
Haha 1900 pages on "basics"? I suspect there are better resources for each topic covered (e.g Gilbert Strang books and OCW lectures for Linear Algebra), but it is definitely interesting to peruse and get a sense of relevant topics.
abhisuri97 7y ago
I love professor gallier! He's an incredible person. That being said, this is faaaaaar beyond basics. It'd be more appropriate to call this an incomplete (aiming to be comprehensive) guide to almost everything you need to know in computer science (related to math).
- graycat 7y ago
  My look at the table of contents looked like the book is short on both probability and statistics.
jointpdf 7y ago
“Math Basics” is quite the misnomer—it gives the impression that one would need to study all of the contents of this book to be an effective practitioner in CS or ML. Memorizing every definition and theorem in this book would be neither necessary nor sufficient for that purpose. Keep in mind it can take an hour, and sometimes way more, to really absorb a single page of a math book like this (do the math). This is more of a reference text.
- PopeDotNinja 7y ago
  This is book 1962 pages long. If this is basic, how long is the advanced book?!
  - langitbiru 7y ago
    It reminds me of Introduction to Algorithms, a classic book by Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest. It is over 1000 pages long. But they call it "Introduction..." :)
    - fesoliveira 7y ago
      To be honest, Cormen et Al. is an actual introductory book. While it is humongous, it covers all of the basic math, logic and algorithms you see in the first 2 or 3 years of an computer science undergraduate course. It is an introduction in the sense that knowing and mastering the tools that the book provides you will set you up for more advanced topics in the many areas of computer science. For instance, a big number of important algorithms in machine learning, computational geometry and other topics use the basic strategies of greedy, divide-and-conquer or dynamic programming algorithms.
  - Koshkin 7y ago
    Actually. many "advanced" texts are rather short (perhaps because they are more specialized or they do not need to be verbose or "entertaining" like many elementary texts are).
    - 7y ago
j7ake 7y ago
Even if you were ambitious and manage to read 5 pages per day everyday this would take you more than one year to read this from start to finish.
amthewiz 7y ago
Basics should be concepts that get you to 80% and tell you where to look for the rest 20%. This book tries to get you directly to 95% and is best treated as a reference book.
manca 7y ago
Wow!
laichzeit0 7y ago
Strangely, probability theory is completely omitted.
- kebman 7y ago
  Is there a probable cause?
TimMurnaghan 7y ago
Nice to see wavelets in here - but it's a shame that he seems to be encouraging people to actually use Haar wavelets. They're fine for teaching - but there are usually better choices in real life. Daubechies are a good default
emmanueloga_ 7y ago
In basic calculus one can burn countless hours memorizing mechanical rules to derive and integrate different function forms, or one can just plug the function into something like wolfram-alpha and get, for a lot of useful cases, a symbolic answer, or at least some approximate answer for a point or interval. The point is, understanding integrals and derivatives doesn't require one to memorize all the mechanical rules. Using software to compute those functions can be a huge time saver. No one should go with pen an paper double checking if that polynomial integral is correct or not! With a book almost 2000 pages long, I wonder if this books leans more heavily on the mechanical-rules side of math. In my mind, is the difference between writing a book such that you can write your own wolfram alpha, or writing a book so you can just use it.
- MAXPOOL 7y ago
  You don't need to memorize rules when studying math. Just like you don't need to spend any time to memorize syntax for programming languages. You automatically remember things you use a lot. Once you have spent countless hours doing exercises to the extent that you understand the math, you already remember the rules. If you have not spent countless hours doing exercises, you don't understand anything at this level. You don't hire a programmer who has read all the books and 'understands' programming but has never programmed. It's the same with math. You don't just read a math book from start to finish. You can use wolfram alpha for visualizing functions, not for learning math.
  - plinkplonk 7y ago
    + 100. I can't upvote this enough. Programmers have spent countless hours practising programming to the point where they have forgotten how difficult it was in the beginning. A non programmer might think of programming as "memorizing hundreds of rules" to get anything done, but one doesn't learn programming by sitting around explicitly memorizing hundreds of rules and then begin to program. Actually writing programs with a minimal set of 'rules' memorized and then adding more as needed is how one typically learns programming.
    - 7y ago
meuk 7y ago
Whoah, this covers a lot. I was expecting some linear algebra, calculus, and discrete math, but there's actually some stuff in there I don't know after doing a masters in math.
- djaychela 7y ago
  That makes me feel somewhat better - I saw the title of 'Maths Basics' and thought 'Great!'... then I saw it's 1,962 pages - if that's the basics, how much is the intermediate and advanced bit?
- raxxorrax 7y ago
  Thought the same. 2000 pages - Basics was probably an understatement... Just looked at a few pages and it seems really illustrative. I am just a light-weight mathematician as a computer scientist, but I really would have liked such a comprehensive script for studying. I hate it when profs reduce everything to minimal definitions and expect studends to make sense of it. There are countless books but it is always a gamble that they focus on the topic at hand and don't suffer from the same problems. This even gives you "motivational examples" which are extremely helpful for comprehension in my opinion.
ps101 7y ago
I really can't figure out who the target audience for this book is, if it has a target audience at all.
sgt101 7y ago
Math "Basics" in nearly 2k pages!
estomagordo 7y ago
Okay, so this looks potentially awesome. But given that it is a reference work rather than some introductory "basic" little quick read-through, I'd prefer to have it in paper form. Any hope of that happening?
- deleted 7y ago
  [deleted]
jvehent 7y ago
"Math Basics" It's 2000 pages long....
- kouh 7y ago
  Chapter 1 can't be more basic, a great summary of what will remain intuitively after reading the whole opus
  - euph0ria 7y ago
    lol
ForFreedom 7y ago
Q: Is this all necessary for ML?
- ovi256 7y ago
  No, you can be an ML practitioner with just an intuitive understanding of, say, gradient descent works and you would do fine. You can even pick up that intuitive understanding on a strictly need-to-know basis, when it's needed for learning an ML technique. That's what fast.ai teaches. For being more than a practitioner, like an implementer of new ML libraries or a researcher, of course you'd need to know more.
- TrackerFF 7y ago
  No, but there are some good fundamentals. I.e, The optimization bit for when dealing with SVM, Kernel methods, etc. All parts that college ML courses cover in depth
xiaolingxiao 7y ago
The professor who wrote this is Jean Gallier, and I had him for advanced linear algebra at Penn. I am also pretty close to him in so far as a student can be close to a professor. On a personal note, he is one of the funniest professor I've had, and all math professors are characters. For the people who are interested in ML, the thing to remember here is that he is a Serious mathematician, and he values rigor and in-depth understanding above all. A lot of his three star homework problems were basically impossible. He writes books first and foremost so he can understand things better. In math books, there's the book you first read when you don't understand something, then the book you read when you understand everything. This is book in the link. for linear algebra, this:https://www.amazon.com/Introduction-Linear-Algebra-Gilbert-Strang/dp/0980232775/ref=asc_df_0980232775/?tag=hyprod-20&linkCode=df0&hvadid=312152840806&hvpos=1o1&hvnetw=g&hvrand=13794249926302782300&hvpone=&hvptwo=&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=9015292&hvtargid=pla-454800779501&psc=1&tag=&ref=&adgrpid=61316181319&hvpone=&hvptwo=&hvadid=312152840806&hvpos=1o1&hvnetw=g&hvrand=13794249926302782300&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=9015292&hvtargid=pla-454800779501 https://www.amazon.com/Introduction-Linear-Algebra-Gilbert-S...)
- jimbokun 7y ago
  As someone who never took an under graduate linear graduate course, videos of Strang's lectures got me through a couple graduate machine learning courses. https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-lectures/ https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra... What he does with chalk and a blackboard is far more effective than anything done today with Powerpoint, fancy computer animations, or what have you.
- xiaolingxiao 7y ago
  Apparently, you guys are overloading the university's server! Nice job guys!!
currymj 7y ago
this is an incredible reference for a machine learning researcher who wants to fill in some gaps in their existing mathematical knowledge. But I would be shocked if this would be of any use for someone trying to learn a little linear algebra in order to play with neural networks. For that I think you still want Strang. I think "foundations" might have been a better word than "basics" here. "Basics" in any case is not in the printed title, only in the filename.
floki999 7y ago
The writing style of this book i.e. rigorous math notation and proposition/proof presentation is going to put off the great majority of potential CS and ML readers. At almost 2000 pages it sure makes a great door-stop though.
impaktdevices 7y ago
Me: Oh, good! I've always been pretty good at math but I want to learn how to make sense of the math I encounter in CS and ML. [Reads the first paragraph of the 2nd chapter] Me: I don't know anything about math. At all.
decotz 7y ago
403 forbidden. Can someone host this?
- ccffph 7y ago
  https://web.archive.org/web/20190730230113/https://www.cis.upenn.edu/~jean/math-basics.pdf https://web.archive.org/web/20190730230113/https://www.cis.u...
parasdahal 7y ago
403 forbidden, can someone help us out?
- itchyjunk 7y ago
  It now redirects to google drive. Simply refresh. It redirects to [0]. [0] https://drive.google.com/file/d/1sJvLQwxMyu89t2z4Zf9tD7O7efnbIUyB/view https://drive.google.com/file/d/1sJvLQwxMyu89t2z4Zf9tD7O7efn...
ppcdeveloper 7y ago
This is nice.
strikelaserclaw 7y ago
This is more like courses a talented undergrad math major would take through 4 years.
gantkimthis 7y ago
is Linear Algebra something you need to work with machine learning?
iserlohnmage 7y ago
Can someone recommend me a book on Linear Algebra, Statistics and Probability?
- neighbour 7y ago
  Second this. If it has exercises, even better. About to undertake my MSc Comp Sci and would like to learn Discrete Math as I'm coming from a non-math background (did B.IT at school and only took two math courses).
- bobby358 7y ago
  I really like Linear Algebra Done Right by Axler

883 points by oldgun 7y ago

planetabhi 7y ago

This is a good book

tempodox 7y ago

This book looks great, I've been looking for a resource like that.

merlinsbrain 7y ago

I love that they have problems you can solve as well at the end of (almost) every chapter. This IS a lot of math (1,962 pages) and it’s missing a preface/introduction which would have been helpful to understand if I need to go linear or if a la carte is okay. At the moment I’d assume each major section is independent. Awesome find! Wonder how It’s used. (One of) the author(s) seems pretty prolific too - http://www.cis.upenn.edu/~jean/ http://www.cis.upenn.edu/~jean/

Koshkin 7y ago
> a la carte Yeah, I wish we had an online resource (other than Wikipedia) anyone could learn any sort of math from in a systematic way... Oh well.
- MikeTheGreat 7y ago
  I'm wondering - is this a genuine request, or a snarky, implicit reference to an online resource for learning math somewhere? I'd love to know about the existing resource, if it exists. (The only thing that comes to mind is Wolfram Alpha, which didn't seem 'systematic' the last time I skimmed the main page)
  - house9-2 7y ago
    Maybe they were referring to Khan Academy? https://www.khanacademy.org/math https://www.khanacademy.org/math
    - pirocks 7y ago
      I think Khan academy for higher math/physics/chemistry /CS would be invaluable. Maybe it could be crowdsourced in some way?
      - kebman 7y ago
        Well, there's always 3Blue1Brown: https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw Take a look at his course, Essence of Calculus: https://www.youtube.com/watch?v=WUvTyaaNkzM&list=PLZHQObOWTQDMsr9K-rj53DwVRMYO3t5Yr https://www.youtube.com/watch?v=WUvTyaaNkzM&list=PLZHQObOWTQ... I'm not fluent in math, but I find it fascinating to watch, for some reason. NancyPi is also great: https://www.youtube.com/channel/UCRGXV1QlxZ8aucmE45tRx8w https://www.youtube.com/channel/UCRGXV1QlxZ8aucmE45tRx8w Oh, and don't forget the Mathologer: https://www.youtube.com/channel/UC1_uAIS3r8Vu6JjXWvastJg https://www.youtube.com/channel/UC1_uAIS3r8Vu6JjXWvastJg What a great guy! While I'm at it, go watch Numberphile too: https://www.youtube.com/user/numberphile https://www.youtube.com/user/numberphile Is there anyone I forgot? :D
- gonza 7y ago
  The way i'm trying to learn math right now is with https://ocw.mit.edu https://ocw.mit.edu, then you can look up into CS or Math programs to progress with other courses
- bigred100 7y ago
  https://courses.maths.ox.ac.uk/overview/undergraduate#37105 https://courses.maths.ox.ac.uk/overview/undergraduate#37105 Oxfords course stuff provides some structure to the interested user
- graycat 7y ago
  What would you like to know? Ask and here you might receive ....
deleted 7y ago
[deleted]
piggybox 7y ago
Every time I came across a new book on HN, I feel more needs to be done in my rest of life :)

Gene_Parmesan 7y ago

From the start of Chapter 2: "In the following four chapters, the basic algebraic structures (groups, rings, fields, vectorspaces) are reviewed, with a major emphasis on vector spaces. Basic notions of linear algebra such as vector spaces, subspaces, linear combinations, linear independence, [...], dual spaces,hyperplanes, transpose of a linear maps, are reviewed." If anyone needs to start even earlier than this, I've actually found "3D Math Basics for Graphics and Game Development" to be a good true intro for linear algebra-related stuff. I think this would probably hold even if your primary interest is something other than graphics/game dev. Some of the text in that book's intro is a little cringey with its reliance on kind of juvenile game references, but I didn't find that sort of writing continuing during the actual text. So just push past that stuff. I got a copy of it to act as a refresher before diving into Real-Time Collision Detection since it's been quite a long time since formal math for me (as in, high school, because I'm self-taught in CS). I've managed to make up a lot of ground by working hard and finding classes to audit online (Strang's linear alg course on OCW is a good one), but I have found that depressingly few math texts which claim to be "introductory" are actually truly introductory. This isn't a slight against the linked work, I absolutely love when profs make resources such as this freely available. "How to Prove It" and "Book of Proof" are also great intros to formal math, if less immediately practical.

hx2a 7y ago
> If anyone needs to start even earlier than this, I've actually found "3D Math Basics for Graphics and Game Development" to be a good true intro for linear algebra-related stuff. Did you mean to write "3D Math Primer for Graphics and Game Development" [1]? If you did, I agree 100%. I got a lot out of this book and was able to put it to good use for several projects. [1] https://www.amazon.com/Math-Primer-Graphics-Game-Development-ebook/dp/B008KZU548/ https://www.amazon.com/Math-Primer-Graphics-Game-Development...
codesushi42 7y ago
I would disagree about the gamedev book reference, unless you are referring to the real basics of linear algebra. The really important concepts for ML are least squares, eigenvalues and vectors, and SVD. Those concepts are not very relevant to game programming. Well, least squares can be solved with projection, which is relevant for converting between coordinate spaces. But game dev isn't going to give you that intuition.

melodrama 7y ago

Great book. I think it's a good time to mention a couple of nice books (related) 1. Elementary intro to math of machine learning [0]. Its style is a bit less austere than that of OP's. It also has a chapter on probability. It could possible serve as a great prequel to the book linked in the OP. 2. The book on probability related topics of general data science: high-dimensional geometry, random walks, Markov chains, random graphs, various related algorithms etc [1] 3. Support for people who'd like to read books like the one linked in the OP, but never seen any kind of higher math before [2]. This book has a cover that screams trashy book extremely skimpy on actual info (anyone who reads a lot of tech books knows what I am talking about), but surprisingly,it contains everything it says it does and in great detail. Not even actual math textbooks (say, Springer) are usually written with this much detail. Author likes to add bullet point style elaboration to almost every definition and theorem which is (almost) never the case with gazillions of books usually titled "Abstract Algebra", "Real Analysis", "Complex Analysis" etc. Some such books sometimes attach words like "friendly" to their title (say, "Friendly Measure Theory For Idiots") and still do not rise to the occasion. Worse yet, a ton (if not most) of these books are exact clones of each other with different author names attached. The linked book doesn't suffer from any of these problems. [0] Mathematics For Machine Learning by Deisentoth, Faisal, Ong https://mml-book.github.io/book/mml-book.pdf https://mml-book.github.io/book/mml-book.pdf [1] Foundations Of Data Science By Blum, Hopcroft, Kannan http://www.cs.cornell.edu/jeh/book%20no%20so;utions%20March%202019.pdf http://www.cs.cornell.edu/jeh/book%20no%20so;utions%20March%... 2] Pure Mathematics for Beginners: A Rigorous Introduction to Logic, Set Theory, Abstract Algebra, Number Theory, Real Analysis, Topology, Complex Analysis, and Linear Algebra by Steve Warner https://www.amazon.com/Pure-Mathematics-Beginners-Rigorous-Introduction/dp/0999811754 https://www.amazon.com/Pure-Mathematics-Beginners-Rigorous-I...

iamcreasy 7y ago
I'll check out the last one. Currently I am teaching myself real analysis.
deleted 7y ago
[deleted]

bigred100 7y ago

That’s... quite a lot of math

markus_zhang 7y ago

That's almost 2,000 pages of math...I don't know why and how, but somehow I forgot most of the Statistics knowledge I obtained as a graduate student (in Stat) 10 years ago. I remembered that I took an advanced course about Bayesian Inference, and one course about Multivariate Statistics (PCA, Factor analysis, these kind of things), and my project is about Bernstein Polynomial. That's it...

xenihn 7y ago
You forget complex things that you don't use regularly. Math, spoken languages, written languages, coding...of course you can re-learn it, and re-learning is faster than learning it for the first time. Based on speaking to my managers in the past, it seems like a year-long lapse is enough for you to lose an incredible amount of retained knowledge/skill. But it's not a permanent loss.
- markus_zhang 7y ago
  Yeah agreed, sometimes reading a research paper from the DS team would actually ring a bell somewhere and I know where to look at. I'm re-learning Statistics from bottom up at the moment lol but this 2,000-page book really looks daunting. I'm pretty sure I didn't take any advanced optimization course back in university.

SKILNER 7y ago

Very first sentence of 2.1 is full of notation, symbols and terms that I, as a prospective student, might not understand. So many teachers seem incapable of stepping outside their sphere of knowledge and seeing what they know and others do not. And so much work went into this.

lone_haxx0r 7y ago
On the other hand, it's a perfect refresher for those of us who "know" this, but somehow forgot most of it.
vonholstein 7y ago
As someone in the same boat I've found this book to be very helpful. https://www.amazon.com/gp/product/1466230525 https://www.amazon.com/gp/product/1466230525 - Mathematical Notation: A Guide for Engineers and Scientists
- alexbanks 7y ago
  I purchased this. I've been trying to brush up on CS fundamentals (it's been a long time since college), but I get stuck just on trying to understand what I'm being asked to learn. Thank you
bradlys 7y ago
Even as someone who did their undergrad in math, it's a bit tough to digest. I'd have to look up a lot of the terms again. It's been five years since I was in college. Shows how little I've used it all since I graduated. It definitely looks more like, "math basics" for x field. Kinda like "automotive basics" for Honda or Ford vehicles. Where it's presumed that you know a lot of automotive lingo to begin with and you just need to know what spark plugs go with what engine. And not, "what is a spark plug?"
graycat 7y ago
Here's what he is doing. He wants to start with the set of real numbers, intuitively the points on the line, usually denoted by R, maybe typed in some special font. Then he wants to define, say, addition of real numbers. So, given two real numbers, x and y, that might be equal, he wants to define x + y. So, here he wants to regard addition, that is, +, as an operation. Then, as is usual for defining operations, he wants an operation to be just a special case of a function. So, he wants to call + a function. So, + will be a function of two variables, say, x and y. With usual function notation we will have +(x,y) = x + y The set of all (x,y) is the domain of the function, and the set of all x + y is the range. So, that defines the function + except commonly in pure math we want to be explicit about the range and domain of the function. For function +, the range is just the set of all pairs (x,y) with x and y in R. That set is also the set theory Cartesian product of set R with itself and written R x R. So, the domain of + is R x R. The range is just R. Then to be explicit about the range and domain of function +, we can write +: R x R --> R which says that + is a function with range R x R and domain R. We learned how to add in, what, kindergarten? So, why make this so complicated? Well, he wants to regard the real numbers as just one example of lots of different algebraic systems, e.g., groups, fields, vector spaces, and much more, with lots of operations and, possibly, more that could be defined. E.g., later in his book he will want to add vectors and matrices, take an inner product of two vectors, and multiply two matrices. So, back to addition on the real numbers, he wants to regard that as just a special case of an operation on an algebraic system. IMHO there's not much benefit for making adding two real numbers look so complicated. Whatever he did in that chapter for defining addition on the reals, soon he is discussing matrix multiplication with no definition at all -- assuming the reader already understands that, that is defined and discussed many pages later in his book. So, in his notation +: R x R --> R and matrix multiplication, he is using material before he has defined it, even before he has motivated, explained, exemplified, indicated the value of, and defined it. In good math writing and in good technical writing more generally, that practice is, in non-technical language, a bummer. But from the table of contents, it appears that the book has quite a long list of possibly interesting narrow topics. And maybe for the routine material, his proofs and presentation are good -- maybe. I thought enough of the book to keep a copy of the PDF. It's there; if someday I want a discussion of some narrow topic, maybe I'll try his book! In mathematical writing, it used to be common for the word processing to be much more work than the mathematics! Now with TeX and LaTeX, and I'm assuming that the book used one of these two, the flood gates are open!

mjortberg521 7y ago

Great to see Prof. Gallier featured on here!

jaimex2 7y ago

I look at this and profoundly thank the people who make ml libraries for us the rest of us.

vecter 7y ago
The vast vast vast vast vast majority of this book (which is more of a reference and encyclopedia than an actual book for learning) is not required for implementing most ML libraries.

krosaen 7y ago

Haha 1900 pages on "basics"? I suspect there are better resources for each topic covered (e.g Gilbert Strang books and OCW lectures for Linear Algebra), but it is definitely interesting to peruse and get a sense of relevant topics.

abhisuri97 7y ago

I love professor gallier! He's an incredible person. That being said, this is faaaaaar beyond basics. It'd be more appropriate to call this an incomplete (aiming to be comprehensive) guide to almost everything you need to know in computer science (related to math).

graycat 7y ago
My look at the table of contents looked like the book is short on both probability and statistics.

jointpdf 7y ago

“Math Basics” is quite the misnomer—it gives the impression that one would need to study all of the contents of this book to be an effective practitioner in CS or ML. Memorizing every definition and theorem in this book would be neither necessary nor sufficient for that purpose. Keep in mind it can take an hour, and sometimes way more, to really absorb a single page of a math book like this (do the math). This is more of a reference text.

PopeDotNinja 7y ago
This is book 1962 pages long. If this is basic, how long is the advanced book?!
- langitbiru 7y ago
  It reminds me of Introduction to Algorithms, a classic book by Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest. It is over 1000 pages long. But they call it "Introduction..." :)
  - fesoliveira 7y ago
    To be honest, Cormen et Al. is an actual introductory book. While it is humongous, it covers all of the basic math, logic and algorithms you see in the first 2 or 3 years of an computer science undergraduate course. It is an introduction in the sense that knowing and mastering the tools that the book provides you will set you up for more advanced topics in the many areas of computer science. For instance, a big number of important algorithms in machine learning, computational geometry and other topics use the basic strategies of greedy, divide-and-conquer or dynamic programming algorithms.
- Koshkin 7y ago
  Actually. many "advanced" texts are rather short (perhaps because they are more specialized or they do not need to be verbose or "entertaining" like many elementary texts are).
  - 7y ago

j7ake 7y ago

Even if you were ambitious and manage to read 5 pages per day everyday this would take you more than one year to read this from start to finish.

amthewiz 7y ago

Basics should be concepts that get you to 80% and tell you where to look for the rest 20%. This book tries to get you directly to 95% and is best treated as a reference book.

manca 7y ago

Wow!

laichzeit0 7y ago

Strangely, probability theory is completely omitted.

kebman 7y ago
Is there a probable cause?

TimMurnaghan 7y ago

Nice to see wavelets in here - but it's a shame that he seems to be encouraging people to actually use Haar wavelets. They're fine for teaching - but there are usually better choices in real life. Daubechies are a good default

emmanueloga_ 7y ago

In basic calculus one can burn countless hours memorizing mechanical rules to derive and integrate different function forms, or one can just plug the function into something like wolfram-alpha and get, for a lot of useful cases, a symbolic answer, or at least some approximate answer for a point or interval. The point is, understanding integrals and derivatives doesn't require one to memorize all the mechanical rules. Using software to compute those functions can be a huge time saver. No one should go with pen an paper double checking if that polynomial integral is correct or not! With a book almost 2000 pages long, I wonder if this books leans more heavily on the mechanical-rules side of math. In my mind, is the difference between writing a book such that you can write your own wolfram alpha, or writing a book so you can just use it.

MAXPOOL 7y ago
You don't need to memorize rules when studying math. Just like you don't need to spend any time to memorize syntax for programming languages. You automatically remember things you use a lot. Once you have spent countless hours doing exercises to the extent that you understand the math, you already remember the rules. If you have not spent countless hours doing exercises, you don't understand anything at this level. You don't hire a programmer who has read all the books and 'understands' programming but has never programmed. It's the same with math. You don't just read a math book from start to finish. You can use wolfram alpha for visualizing functions, not for learning math.
- plinkplonk 7y ago
  + 100. I can't upvote this enough. Programmers have spent countless hours practising programming to the point where they have forgotten how difficult it was in the beginning. A non programmer might think of programming as "memorizing hundreds of rules" to get anything done, but one doesn't learn programming by sitting around explicitly memorizing hundreds of rules and then begin to program. Actually writing programs with a minimal set of 'rules' memorized and then adding more as needed is how one typically learns programming.
  - 7y ago

meuk 7y ago

Whoah, this covers a lot. I was expecting some linear algebra, calculus, and discrete math, but there's actually some stuff in there I don't know after doing a masters in math.

djaychela 7y ago
That makes me feel somewhat better - I saw the title of 'Maths Basics' and thought 'Great!'... then I saw it's 1,962 pages - if that's the basics, how much is the intermediate and advanced bit?
raxxorrax 7y ago
Thought the same. 2000 pages - Basics was probably an understatement... Just looked at a few pages and it seems really illustrative. I am just a light-weight mathematician as a computer scientist, but I really would have liked such a comprehensive script for studying. I hate it when profs reduce everything to minimal definitions and expect studends to make sense of it. There are countless books but it is always a gamble that they focus on the topic at hand and don't suffer from the same problems. This even gives you "motivational examples" which are extremely helpful for comprehension in my opinion.

ps101 7y ago

I really can't figure out who the target audience for this book is, if it has a target audience at all.

sgt101 7y ago

Math "Basics" in nearly 2k pages!

estomagordo 7y ago

Okay, so this looks potentially awesome. But given that it is a reference work rather than some introductory "basic" little quick read-through, I'd prefer to have it in paper form. Any hope of that happening?

deleted 7y ago
[deleted]

jvehent 7y ago

"Math Basics" It's 2000 pages long....

kouh 7y ago
Chapter 1 can't be more basic, a great summary of what will remain intuitively after reading the whole opus
- euph0ria 7y ago
  lol

ForFreedom 7y ago

Q: Is this all necessary for ML?

ovi256 7y ago
No, you can be an ML practitioner with just an intuitive understanding of, say, gradient descent works and you would do fine. You can even pick up that intuitive understanding on a strictly need-to-know basis, when it's needed for learning an ML technique. That's what fast.ai teaches. For being more than a practitioner, like an implementer of new ML libraries or a researcher, of course you'd need to know more.
TrackerFF 7y ago
No, but there are some good fundamentals. I.e, The optimization bit for when dealing with SVM, Kernel methods, etc. All parts that college ML courses cover in depth

xiaolingxiao 7y ago

The professor who wrote this is Jean Gallier, and I had him for advanced linear algebra at Penn. I am also pretty close to him in so far as a student can be close to a professor. On a personal note, he is one of the funniest professor I've had, and all math professors are characters. For the people who are interested in ML, the thing to remember here is that he is a Serious mathematician, and he values rigor and in-depth understanding above all. A lot of his three star homework problems were basically impossible. He writes books first and foremost so he can understand things better. In math books, there's the book you first read when you don't understand something, then the book you read when you understand everything. This is book in the link. for linear algebra, this:https://www.amazon.com/Introduction-Linear-Algebra-Gilbert-Strang/dp/0980232775/ref=asc_df_0980232775/?tag=hyprod-20&linkCode=df0&hvadid=312152840806&hvpos=1o1&hvnetw=g&hvrand=13794249926302782300&hvpone=&hvptwo=&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=9015292&hvtargid=pla-454800779501&psc=1&tag=&ref=&adgrpid=61316181319&hvpone=&hvptwo=&hvadid=312152840806&hvpos=1o1&hvnetw=g&hvrand=13794249926302782300&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=9015292&hvtargid=pla-454800779501 https://www.amazon.com/Introduction-Linear-Algebra-Gilbert-S...)

jimbokun 7y ago
As someone who never took an under graduate linear graduate course, videos of Strang's lectures got me through a couple graduate machine learning courses. https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-lectures/ https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra... What he does with chalk and a blackboard is far more effective than anything done today with Powerpoint, fancy computer animations, or what have you.
xiaolingxiao 7y ago
Apparently, you guys are overloading the university's server! Nice job guys!!

currymj 7y ago

this is an incredible reference for a machine learning researcher who wants to fill in some gaps in their existing mathematical knowledge. But I would be shocked if this would be of any use for someone trying to learn a little linear algebra in order to play with neural networks. For that I think you still want Strang. I think "foundations" might have been a better word than "basics" here. "Basics" in any case is not in the printed title, only in the filename.

floki999 7y ago

The writing style of this book i.e. rigorous math notation and proposition/proof presentation is going to put off the great majority of potential CS and ML readers. At almost 2000 pages it sure makes a great door-stop though.

impaktdevices 7y ago

Me: Oh, good! I've always been pretty good at math but I want to learn how to make sense of the math I encounter in CS and ML. [Reads the first paragraph of the 2nd chapter] Me: I don't know anything about math. At all.

decotz 7y ago

403 forbidden. Can someone host this?

ccffph 7y ago
https://web.archive.org/web/20190730230113/https://www.cis.upenn.edu/~jean/math-basics.pdf https://web.archive.org/web/20190730230113/https://www.cis.u...

parasdahal 7y ago

403 forbidden, can someone help us out?

itchyjunk 7y ago
It now redirects to google drive. Simply refresh. It redirects to [0]. [0] https://drive.google.com/file/d/1sJvLQwxMyu89t2z4Zf9tD7O7efnbIUyB/view https://drive.google.com/file/d/1sJvLQwxMyu89t2z4Zf9tD7O7efn...

ppcdeveloper 7y ago

This is nice.

strikelaserclaw 7y ago

This is more like courses a talented undergrad math major would take through 4 years.

gantkimthis 7y ago

is Linear Algebra something you need to work with machine learning?

iserlohnmage 7y ago

Can someone recommend me a book on Linear Algebra, Statistics and Probability?

neighbour 7y ago
Second this. If it has exercises, even better. About to undertake my MSc Comp Sci and would like to learn Discrete Math as I'm coming from a non-math background (did B.IT at school and only took two math courses).
bobby358 7y ago
I really like Linear Algebra Done Right by Axler