5 ms·

Did the guy ever deliver on his promise to create a better system? I thought the problems he was pointing out was motivation for a better system (that united t

Did the guy ever deliver on his promise to create a better system? I thought the problems he was pointing out was motivation for a better system (that united the naive understanding and rigor better?), but the further into the article I got, the more it just seemed like a list of complaints. Having 1) studied "naive statistics" when I was young and 2) learning the full-on measure-theory version a la Bourbaki, and now 3) using applied statistics every day, I can say the naive version is the most useful by far. Yes, there is hand waving, but the hand waving works, and you really don't get into that much trouble unless you try to be pathological. I admit, it would be great to have a version where the naive intuition (many of which are motivated by empirics, physics, and real world situations) and the theoretical definition matched better. Even still, I don't think the formal treatment is appropriate as an introduction -- it's an unnecessary hazing on learning minds when they can get most of the value with a bit of hand waving.

wjn0 8y ago
Interesting that you chose statistics as your example. I've always considered statistics to be the one branch of math I've studied where intuition keeps pace with depth and rigour. The caveat being that I am by definition an amateur when it comes to any other branch of math.
psykotic 8y ago
"Measure theory a la Bourbaki" is probably not what you meant if you did probability theory and statistics. Bourbaki famously sidestepped classical measure theory with sigma algebras by constructing Radon measures as functionals via functional analysis. This is sufficient for many purposes but not for probability theory. First and foremost, sigma algebras in probability theory are not just a technical device to avoid paradoxes with unmeasurable sets. Sigma algebras play a starring role in conditional expectation, stochastic processes and martingale theory. The best way to appreciate the information-theoretic role of sigma algebras is to look at them in the simplest case, where you have a discrete-time, finite-valued process. Then a sigma algebra is equivalent to a partition of the state space and it represents the information that can be gained from an observation; it's like a random variable without specific values, just the discriminating information from different outcomes. To say that a random variable is measurable with respect to the sigma algebra is to say that its value may only depend on information that can be gained from an observation. A filtration of sigma algebras corresponds to a causal series of observations where the observer learns more information over time. The conditional expectation of a random variable with respect to a sigma algebra (or partition or other random variable) is another random variable that tells you the expectation over the states consistent with a given observation; this new random variable is measurable with respect to the sigma algebra you conditioned on, which as mentioned earlier means it only depends on the information gained from an observation. The conditional expectation is the best least-squares estimator given the information from an observation in the same way that the usual expectation is the best least-squares estimator given no information.
- heyitsguay 8y ago
  As someone who moved from pure mathematics -> applied mathematics -> machine learning, perhaps I can offer a similar perspective to GP that explains why the measure theory stuff might not seem so useful to some people. Basically, in application you never need to worry about anything but your simplest case - that of a discrete-time, finite-valued process. All of the subtleties of measure theory, which occupy most of what mathematical measure theorists/analysts work on, are introduced when you assume you've got an infinity or a continuum somewhere in a model, and that assumption is never borne out in practice. Unless you really want to do theoretical math, you're better off learning only the aspects that relate to finite functions, i.e. almost none of what a real mathematician would consider to be "math". And that's not just out of laziness or disinterest - once you've restricted yourself to finite functions for empirical applications, you encounter a whole new realm of engineering difficulties to deal with, that theoretical mathematics doesn't touch. You've got to learn how to solve a different class of problems that aren't as purely logical, but are every bit as challenging. To put it another way - think about all the pathologies you may encounter working with even a "nice" function space like L^2(R). You'll never deal with those pathologies in reality, because every empirical function is much better behaved - finite domain, finite range, and even if you assume a continuous domain, you can choose a model that only has a finite number of discontinuities, is Lipschitz continuous in between them, has finite total variation, etc. And that's why the hand-wavey, "intuitive" approach works so well if you're not a theorist, at least in my opinion.
  - psykotic 8y ago
    For what it's worth, my professional background is entirely in game development and systems programming. I'm not a professional academic. All the professional work I've done with mathematics is applied. I think you're overstating some things, but I mostly agree. My main disagreement is with your implicit premise that the best practical theory should exist at the same level of abstraction as practical applications. The real numbers have been a really successful practical theory. Physicists and applied mathematicians know they don't "really" exist, but more "realistic" alternatives are awkward and messy. The same applies to more extravagant theoretical constructions like Hilbert spaces. They're an extremely nice mathematical setting for applications (e.g. optimal control, approximation theory, finite element methods, quantum mechanics). No-one should be losing much sleep over their ubiquity in applications. If your point is that we shouldn't belabor some of their technical details when teaching them to practitioners, sure, but that's already the case.
hyperpallium 8y ago
He did at least do the MathFoundations youtube videos he mentioned (pity he didn't edit a link in) https://youtube.com/playlist?list=PL5A714C94D40392AB https://youtube.com/playlist?list=PL5A714C94D40392AB

wjn0 8y ago

Interesting that you chose statistics as your example. I've always considered statistics to be the one branch of math I've studied where intuition keeps pace with depth and rigour. The caveat being that I am by definition an amateur when it comes to any other branch of math.

psykotic 8y ago

"Measure theory a la Bourbaki" is probably not what you meant if you did probability theory and statistics. Bourbaki famously sidestepped classical measure theory with sigma algebras by constructing Radon measures as functionals via functional analysis. This is sufficient for many purposes but not for probability theory. First and foremost, sigma algebras in probability theory are not just a technical device to avoid paradoxes with unmeasurable sets. Sigma algebras play a starring role in conditional expectation, stochastic processes and martingale theory. The best way to appreciate the information-theoretic role of sigma algebras is to look at them in the simplest case, where you have a discrete-time, finite-valued process. Then a sigma algebra is equivalent to a partition of the state space and it represents the information that can be gained from an observation; it's like a random variable without specific values, just the discriminating information from different outcomes. To say that a random variable is measurable with respect to the sigma algebra is to say that its value may only depend on information that can be gained from an observation. A filtration of sigma algebras corresponds to a causal series of observations where the observer learns more information over time. The conditional expectation of a random variable with respect to a sigma algebra (or partition or other random variable) is another random variable that tells you the expectation over the states consistent with a given observation; this new random variable is measurable with respect to the sigma algebra you conditioned on, which as mentioned earlier means it only depends on the information gained from an observation. The conditional expectation is the best least-squares estimator given the information from an observation in the same way that the usual expectation is the best least-squares estimator given no information.

heyitsguay 8y ago
As someone who moved from pure mathematics -> applied mathematics -> machine learning, perhaps I can offer a similar perspective to GP that explains why the measure theory stuff might not seem so useful to some people. Basically, in application you never need to worry about anything but your simplest case - that of a discrete-time, finite-valued process. All of the subtleties of measure theory, which occupy most of what mathematical measure theorists/analysts work on, are introduced when you assume you've got an infinity or a continuum somewhere in a model, and that assumption is never borne out in practice. Unless you really want to do theoretical math, you're better off learning only the aspects that relate to finite functions, i.e. almost none of what a real mathematician would consider to be "math". And that's not just out of laziness or disinterest - once you've restricted yourself to finite functions for empirical applications, you encounter a whole new realm of engineering difficulties to deal with, that theoretical mathematics doesn't touch. You've got to learn how to solve a different class of problems that aren't as purely logical, but are every bit as challenging. To put it another way - think about all the pathologies you may encounter working with even a "nice" function space like L^2(R). You'll never deal with those pathologies in reality, because every empirical function is much better behaved - finite domain, finite range, and even if you assume a continuous domain, you can choose a model that only has a finite number of discontinuities, is Lipschitz continuous in between them, has finite total variation, etc. And that's why the hand-wavey, "intuitive" approach works so well if you're not a theorist, at least in my opinion.
- psykotic 8y ago
  For what it's worth, my professional background is entirely in game development and systems programming. I'm not a professional academic. All the professional work I've done with mathematics is applied. I think you're overstating some things, but I mostly agree. My main disagreement is with your implicit premise that the best practical theory should exist at the same level of abstraction as practical applications. The real numbers have been a really successful practical theory. Physicists and applied mathematicians know they don't "really" exist, but more "realistic" alternatives are awkward and messy. The same applies to more extravagant theoretical constructions like Hilbert spaces. They're an extremely nice mathematical setting for applications (e.g. optimal control, approximation theory, finite element methods, quantum mechanics). No-one should be losing much sleep over their ubiquity in applications. If your point is that we shouldn't belabor some of their technical details when teaching them to practitioners, sure, but that's already the case.

hyperpallium 8y ago

He did at least do the MathFoundations youtube videos he mentioned (pity he didn't edit a link in) https://youtube.com/playlist?list=PL5A714C94D40392AB https://youtube.com/playlist?list=PL5A714C94D40392AB