4 ms·

Happy to see a book like this trending on hn, especially with a sentence like: "Using fancy tools like neural nets, boosting, and support vector machines witho

Happy to see a book like this trending on hn, especially with a sentence like: "Using fancy tools like neural nets, boosting, and support vector machines without understanding basic statistics is like doing brain surgery before knowing how to use a band-aid." in it's preface. I definitely agree, since I wasted a lot of time doing fruitless surgeries before I went and learned about band aids in depth. From my look at Part 1, it has some great coverage of the basics, all of which are important. Some of the fundamentals that I see left out are rightly left out since they require experience in real analysis to appreciate, and maybe aren't very actionable. There's few proofs, but, since the goal is a quick understanding, I can also appreciate this. It looks to me like a great intro of statistics for CS people, as the author says.

mathgeek 10y ago
> since I wasted a lot of time doing fruitless surgeries before I went and learned about band aids in depth I'm going to just assume you're being literal here. It's really brightened up my day to imagine the moment such a person discovered you could close up wounds after surgery.
trentmb 10y ago
Suggestions for a more rigorous treatment?
- tezka 10y ago
  My favorites are Casella and Berger: https://books.google.com/books/about/Statistical_Inference.html?id=0x_vAAAAMAAJ https://books.google.com/books/about/Statistical_Inference.h... and Hogg, McKean and Craig: https://www.pearsonhighered.com/program/Hogg-Introduction-to-Mathematical-Statistics-7th-Edition/PGM49624.html https://www.pearsonhighered.com/program/Hogg-Introduction-to... They are both good books. The first one is more rigorous, but the second one covers more breadth and I think has better exercises.
- ilzmastr 10y ago
  Anything with measure theory, though you will regret it :)
  - trentmb 10y ago
    Here's a free one I found in the event anyone else is curious: https://services.math.duke.edu/~rtd/PTE/pte.html https://services.math.duke.edu/~rtd/PTE/pte.html
  - princeb 10y ago
    stochastic calculus really hurt me. i did well for it, I TA'd for it, i look at it from time to time in my work, but I still find it very difficult. i don't think i'm much good at mathematics, i just try very hard and hope for the best.
  - 5olidor 10y ago
    On the probability side, I thought Ash's "Probability & Measure Theory" was good for self-study, although some experience with real analysis is definitely a prerequisite. It can be pretty dense at times, but well worth going through.
- deleted 10y ago
  [deleted]
clishem 10y ago
> "Using fancy tools like neural nets, boosting, and support vector machines without understanding basic statistics is like doing brain surgery before knowing how to use a band-aid." Having studied both statistics and neural networks, I'm not sure if I completely agree with that quote. There are lots of neural network applications that have little to do with statistics (image recognition with convolutional neural networks for example). I am pretty sure that the author means neural networks for statistical applications though.
- DavidSJ 10y ago
  Image classification has everything to do with statistics: you're guessing the probability distribution over the classes conditional on the input image vector; the model is trained through a process of statistical inference (using gradient descent).
  - argonaut 10y ago
    You have stated something that someone with a high school-level (e.g. superficial) understanding of probability/statistics would understand. Most research in neural networks requires only a very superficial level of statistics. There are certainly many areas of neural networks where statistics is important (more theoretical areas), but those don't form the core of the research field. Also, calling (stochastic) gradient descent a form of statistical inference, while technically correct, is a ridiculous stretching of the term. No researcher considers SGD to be a statistical inference algorithm.
- eli_gottlieb 10y ago
  >There are lots of neural network applications that have little to do with statistics (image recognition with convolutional neural networks for example). You're kidding, right? The most fundamental reasons that deep convnets work at all are statistical in nature.
  - niels_olson 10y ago
    Well, yes and no. It's like saying you need to understand Bernoulli's priciple before designing the SR-71. True, but 7 levels short of where you're working.

mathgeek 10y ago

> since I wasted a lot of time doing fruitless surgeries before I went and learned about band aids in depth I'm going to just assume you're being literal here. It's really brightened up my day to imagine the moment such a person discovered you could close up wounds after surgery.

trentmb 10y ago

Suggestions for a more rigorous treatment?

tezka 10y ago
My favorites are Casella and Berger: https://books.google.com/books/about/Statistical_Inference.html?id=0x_vAAAAMAAJ https://books.google.com/books/about/Statistical_Inference.h... and Hogg, McKean and Craig: https://www.pearsonhighered.com/program/Hogg-Introduction-to-Mathematical-Statistics-7th-Edition/PGM49624.html https://www.pearsonhighered.com/program/Hogg-Introduction-to... They are both good books. The first one is more rigorous, but the second one covers more breadth and I think has better exercises.
ilzmastr 10y ago
Anything with measure theory, though you will regret it :)
- trentmb 10y ago
  Here's a free one I found in the event anyone else is curious: https://services.math.duke.edu/~rtd/PTE/pte.html https://services.math.duke.edu/~rtd/PTE/pte.html
- princeb 10y ago
  stochastic calculus really hurt me. i did well for it, I TA'd for it, i look at it from time to time in my work, but I still find it very difficult. i don't think i'm much good at mathematics, i just try very hard and hope for the best.
- 5olidor 10y ago
  On the probability side, I thought Ash's "Probability & Measure Theory" was good for self-study, although some experience with real analysis is definitely a prerequisite. It can be pretty dense at times, but well worth going through.
deleted 10y ago
[deleted]

clishem 10y ago

> "Using fancy tools like neural nets, boosting, and support vector machines without understanding basic statistics is like doing brain surgery before knowing how to use a band-aid." Having studied both statistics and neural networks, I'm not sure if I completely agree with that quote. There are lots of neural network applications that have little to do with statistics (image recognition with convolutional neural networks for example). I am pretty sure that the author means neural networks for statistical applications though.

DavidSJ 10y ago
Image classification has everything to do with statistics: you're guessing the probability distribution over the classes conditional on the input image vector; the model is trained through a process of statistical inference (using gradient descent).
- argonaut 10y ago
  You have stated something that someone with a high school-level (e.g. superficial) understanding of probability/statistics would understand. Most research in neural networks requires only a very superficial level of statistics. There are certainly many areas of neural networks where statistics is important (more theoretical areas), but those don't form the core of the research field. Also, calling (stochastic) gradient descent a form of statistical inference, while technically correct, is a ridiculous stretching of the term. No researcher considers SGD to be a statistical inference algorithm.
eli_gottlieb 10y ago
>There are lots of neural network applications that have little to do with statistics (image recognition with convolutional neural networks for example). You're kidding, right? The most fundamental reasons that deep convnets work at all are statistical in nature.
- niels_olson 10y ago
  Well, yes and no. It's like saying you need to understand Bernoulli's priciple before designing the SR-71. True, but 7 levels short of where you're working.