6 ms·

Looked at the text to be used, Durrett. There's more and with higher quality in any of M. Loeve, J. Neveu, K. Chung, and L. Breiman. For what Tao wrote on his

by graycat 11y ago

Looked at the text to be used, Durrett. There's more and with higher quality in any of M. Loeve, J. Neveu, K. Chung, and L. Breiman. For what Tao wrote on his page about determinism or whatever on that page -- just f'get about that. And what he wrote, confusing a sample space and a probability space, just say that he had a bad day that day.

nkurz 11y ago
And what he wrote, confusing a sample space and a probability space, just say that he had a bad day that day. Could you expand on this? I'm reading his post now, but from the outside, it seems unlikely that Terry got the concepts completely wrong, and more likely that he's using slightly different definitions for the concepts than you are expecting.
- graycat 11y ago
  No, he's just being sloppy. That can be a real pain for students trying to learn. But, be warned: A lot of people work with probability, but only a tiny fraction ever had a course in graduate probability. So, eventually have to learn to put up with, and sometimes rewrite, some of what is written that is not very precise. Here are the accepted definitions: Take a non-empty set, usually denoted by Omega, and call it the sample space or the set of trials. Take a collection of subsets of Omega, usually denoted by script F, and call it the set of events. Have at least enough subsets of Omega in the set of events script F so that script F will be a sigma algebra. So, script F has to have as an element the empty set, be closed under relative complements, and be closed under countable unions. We want all that for probability theory. We want countably infinite so that we can discuss, say, the event that a coin never or always comes up heads. We don't want uncountably infinite because it would create a big mess in the theory. Then the ordered pair (Omega, script F) is called a measurable space -- it doesn't have a measure yet but soon will. On that measurable space, define a positive measure P so that P( Omega ) = 1. Then P is a probability measure on the measurable space. A probability space is the triple ( Omega, script F, P ). Intuitively, a measure assigns to each event A in script F a non-negative real number P(A). A signed measure permits negative values. And of course at times want a measure that yields complex numbers. Measure theory is in, say, P. Halmos, Measure Theory. The back of Halmos has a really nice introduction to probability theory, the Kolmogorov three series theorem in stochastic processes, etc. For more in stochastic processes there is, of course, the classic J. Doob, Stochastic Processes. Halmos was a Doob student and then served as an assistant to von Neumann at the Institute for Advanced Study. From that he wrote Halmos, Finite Dimensional Vector Spaces, a finite dimensional introduction to Hilbert space. Later at University of Chicago Halmos worked in mathematical statistics and made the fundamental contribution to sufficient statistics. IIRC, it was at Chicago that Halmos wrote Measure Theory. Great details on measure theory and much more are in, say, H. Royden, Real Analysis and W. Rudin, Real and Complex Analysis. To narrow measure theory to probability theory, use the famous texts by the authors I listed, Loeve, .... Royden and Rudin are terrific prerequisites to Loeve, etc. Commonly the sigma algebra of events script F is the smallest sigma algebra that contains as a subset a topology, that is, a collection of open sets, on the sample space Omega. That's all standard, beginning stuff in graduate probability -- ah, the only kind should bother with anyway!
  - nkurz 11y ago
    We want all that for probability theory. We want countably infinite so that we can discuss, say, the event that a coin never or always comes up heads. We don't want uncountably infinite because it would create a big mess in the theory. Who should I read for an introduction to that "big mess"? So many of the cases I'm actually interested in require real valued (continuous) inputs and outcomes. While one can quantize these to create an approximation of something countable, it seems like that a much simpler theory would be possible if it was built from the ground up to handle these common non-discrete real-world cases, rather than trying to shoe-horn them into standard probability theory. I was hoping this might be the direction that Terry was headed, with the emphasis on probabilistic methods.
    - jeffcoat 11y ago
      All of this is fine for real-valued inputs and outcomes. It's the number of events (coin flips, measurements, etc) that we're restricting to be countable.
      - graycat 11y ago
        No, the number of events is necessarily also finite or uncountable. Indeed, it is a nice exercise that there are no countably infinite sigma algebras (extra credit for a solution!). It's just can't take uncountably many events, take their union, and assume that the result is also an event.
    - graycat 11y ago
      No problem. For the foundations I outlined, can work just fine with continuous functions, measurable functions, stochastic processes, random variables taking values on the real line, in the complex plane, in finite dimensional real or complex vector spaces with, say, the usual topology, Hilbert and Banach spaces, etc. Can do multi-dimensional Markov processes, and much more. And you can have each point on the real line an event. Fine. But you just can't take the uncountable union of any set of such events and assume that the result is also an event. As for the event a random variable takes a value >= 0? Fine. Or, let the Borel subsets of the real line be the smallest sigma algebra that contains all the open sets, e.g., all the open intervals. Then for Borel set A and real valued random variable X, can ask for the probability X is in A. I believe you will find that you will have a solid foundation for what you want. To see all this stuff, need more than just the sparse definitions and, instead, need an actual text and maybe a course. Recently looked at the on-line materials from MIT and didn't see such a course. Graduate probability is not all that popular in the US; stochastic processes in continuous time is still less popular. To study graduate probability, I'd recommend a good undergraduate major in pure math with good coverage of, say, W. Rudin, Principles of Mathematical Analysis. Then good coverage of linear algebra from more than one of the best known texts. Likely also spend as much time as you can in Halmos, Finite Dimensional Vector Spaces. E.g., at one time, Halmos, Rudin, and Spivak, Calculus on Manifolds were the three main texts for Harvard's famous Math 55. Get good with proving the theorems. I also recommend Fleming, Functions of Several Variables. Then, sure, Royden, Real Analysis. Couldn't be prettier. If not in a hurry, then the real half of Rudin's Real and Complex Analysis. Especially if you like Fourier theory! Then of the probability books, I believe that the nicest, first book is L. Breiman, Probability. He wrote that before he went consulting and came back and did CART and random forests. Next, K. Chung, A Course in Probability Theory. Next, J. Neveu, Mathematical Foundations of the Calculus of Probability. Then, Loeve, Probability Theory. Loeve is huge -- mostly just use it for reference or browse. E.g., it has sufficient statistics and stationary stochastic processes (the EEs love that) IIRC not in the other books. IIRC, both Breiman and Neveu were Loeve students at Berkeley. If do well with Breiman, then for graduate probability, likely can stop there. Else, Chung will then be fast and easy reading and reinforce what you learned in Breiman. Neveu is elegant; my favorite, but deserve extra credit for each workable exercise you can find, not actually work, you understand, just find! Sure, some of the exercises are terrific, half a course in a few lines of an exercise. E.g., he has one of those on statistical decision theory or some such. And see the Tulcea material in the back. Then there's more that you can do on stochastic processes, potential theory via Brownian motion, e.g., for mathematical finance, stochastic optimal control, and more.
nerd_stuff 11y ago
Out of curiosity, what's the background that lets you be so dismissive of a Fields Medal winner? At this point I see a blog post written by a well respected mathematician whom I feel comfortable trusting and it's being brushed aside by I don't know who.
- jonnybgood 11y ago
  Being a great mathematician doesn't necessarily make a great math educator. It is odd Tao chose Durret, but I assume it's due to the book being freely available online.
  - nerd_stuff 11y ago
    As the blog comments suggest there's a difference between a good self-study book and a good textbook for a class. The book will be accompanied by (at least) what look to be a good set of course notes. Some googling around suggests his students are quite pleased with him. He has, according to Wikipedia's intro for him, the undisputed king of math blogs. Both of these point to him being at least a good or above average educator.
- d9fb698e010974b 11y ago
  Well you can just read the reviews to see that the Durrett text isn't well regarded while others like Chung's are. And the criticism about a probability space not being a sample space is correct, but I think it's clear what Tao meant there, namely that the sample space would be a part of a probability space.
  - keithpeter 11y ago
    Thanks The Amazon UK reviews of Chung's book lead me to A Probability Path by Sidney Resnick which appears to be aimed at non-mathematicians. I have invested (speaking as a renegade physicist lacking a systematic exploration of measure theory).
- copperx 11y ago
  Are you advocating "proof by authority "?
deleted 11y ago
[deleted]
olalonde 11y ago
> In the standard foundations of probability theory, as laid out by Kolmogorov, we can then model these events and random variables by introducing a sample space (which will be a probability space) to capture all the ambient sources of randomness; events are then modeled as measurable subsets of this sample space, and random variables are modeled as measurable functions on this sample space. This matches the definitions of Wikipedia pretty closely: "A probability space consists of three parts: A sample space [...], A set of events [...], The assignment of probabilities to the events". So either you misunderstood that sentence or both Wikipedia and Terrence Tao are wrong.
- d9fb698e010974b 11y ago
  The sample space is not the probability space. The probability space has a sample space though, which is what the Wikipedia definition says, but not what Tao literally said. I think it is pretty clear what Tao meant though.
- graycat 11y ago
  " ... a sample space (which will be a probability space)" is, in a word, wrong. The sample space will be part of a probability space but will not "be a probability space".
  - elliptic 11y ago
    Nonsense. Tao is using perfectly idiomatic language here - "The sample space will be a probability space (once we have endowed it with some additional structure)".
    - graycat 11y ago
      That's not good: We don't want to have to use words such as endowed and structure that are not precisely defined in the context and, thus, are conceptually fuzzy. Some intuitive overviews, clearly labeled as such, are fine and can be helpful, but "idiomatic language" just is not. Won't find such in the writings of W. Rudin, P. Halmos, J. Neveu, or any of a long list of authors of some of the best math books. In a good math or computer science journal, a reviewer or the editor would likely reject "idiomatic language". The definitions I gave are the accepted ones.
mturmon 11y ago
To each their own. Durrett's text is perfectly fine. The books you mention are older, and perhaps you originally learned from them, so you still hold a torch? In particular, sending people to Loeve for their first course in measure theoretic probability would be really cruel. Personally, I learned from Feller, and Billingsley, and finally Durrett.
- graycat 11y ago
  > To each their own. Durrett's text is perfectly fine. The books you mention are older, and perhaps you originally learned from them, so you still hold a torch? Durrett seems on most of the topics to have less than any of the four texts I mentioned. > In particular, sending people to Loeve for their first course in measure theoretic probability would be really cruel. I listed the authors in reverse order in which to read them! The easiest start is the last, Breiman. But some things in Loeve are good, e.g., sufficient statistics. Neveu is my favorite as the most elegant, but his exercises are the most difficult. Breiman and Chung are fine. IIRC, both Breiman and Neveu were Loeve students. > Personally, I learned from Feller, and Billingsley, and finally Durrett. I used Feller I and/or II for reference. Otherwise his writing seemed to lack an overall unification. The only Billingsley text I used, and then only for a small topic, was Convergence of Probability Measures.

by graycat 11y ago

nkurz 11y ago

And what he wrote, confusing a sample space and a probability space, just say that he had a bad day that day. Could you expand on this? I'm reading his post now, but from the outside, it seems unlikely that Terry got the concepts completely wrong, and more likely that he's using slightly different definitions for the concepts than you are expecting.

graycat 11y ago
No, he's just being sloppy. That can be a real pain for students trying to learn. But, be warned: A lot of people work with probability, but only a tiny fraction ever had a course in graduate probability. So, eventually have to learn to put up with, and sometimes rewrite, some of what is written that is not very precise. Here are the accepted definitions: Take a non-empty set, usually denoted by Omega, and call it the sample space or the set of trials. Take a collection of subsets of Omega, usually denoted by script F, and call it the set of events. Have at least enough subsets of Omega in the set of events script F so that script F will be a sigma algebra. So, script F has to have as an element the empty set, be closed under relative complements, and be closed under countable unions. We want all that for probability theory. We want countably infinite so that we can discuss, say, the event that a coin never or always comes up heads. We don't want uncountably infinite because it would create a big mess in the theory. Then the ordered pair (Omega, script F) is called a measurable space -- it doesn't have a measure yet but soon will. On that measurable space, define a positive measure P so that P( Omega ) = 1. Then P is a probability measure on the measurable space. A probability space is the triple ( Omega, script F, P ). Intuitively, a measure assigns to each event A in script F a non-negative real number P(A). A signed measure permits negative values. And of course at times want a measure that yields complex numbers. Measure theory is in, say, P. Halmos, Measure Theory. The back of Halmos has a really nice introduction to probability theory, the Kolmogorov three series theorem in stochastic processes, etc. For more in stochastic processes there is, of course, the classic J. Doob, Stochastic Processes. Halmos was a Doob student and then served as an assistant to von Neumann at the Institute for Advanced Study. From that he wrote Halmos, Finite Dimensional Vector Spaces, a finite dimensional introduction to Hilbert space. Later at University of Chicago Halmos worked in mathematical statistics and made the fundamental contribution to sufficient statistics. IIRC, it was at Chicago that Halmos wrote Measure Theory. Great details on measure theory and much more are in, say, H. Royden, Real Analysis and W. Rudin, Real and Complex Analysis. To narrow measure theory to probability theory, use the famous texts by the authors I listed, Loeve, .... Royden and Rudin are terrific prerequisites to Loeve, etc. Commonly the sigma algebra of events script F is the smallest sigma algebra that contains as a subset a topology, that is, a collection of open sets, on the sample space Omega. That's all standard, beginning stuff in graduate probability -- ah, the only kind should bother with anyway!
- nkurz 11y ago
  We want all that for probability theory. We want countably infinite so that we can discuss, say, the event that a coin never or always comes up heads. We don't want uncountably infinite because it would create a big mess in the theory. Who should I read for an introduction to that "big mess"? So many of the cases I'm actually interested in require real valued (continuous) inputs and outcomes. While one can quantize these to create an approximation of something countable, it seems like that a much simpler theory would be possible if it was built from the ground up to handle these common non-discrete real-world cases, rather than trying to shoe-horn them into standard probability theory. I was hoping this might be the direction that Terry was headed, with the emphasis on probabilistic methods.
  - jeffcoat 11y ago
    All of this is fine for real-valued inputs and outcomes. It's the number of events (coin flips, measurements, etc) that we're restricting to be countable.
    - graycat 11y ago
      No, the number of events is necessarily also finite or uncountable. Indeed, it is a nice exercise that there are no countably infinite sigma algebras (extra credit for a solution!). It's just can't take uncountably many events, take their union, and assume that the result is also an event.
  - graycat 11y ago
    No problem. For the foundations I outlined, can work just fine with continuous functions, measurable functions, stochastic processes, random variables taking values on the real line, in the complex plane, in finite dimensional real or complex vector spaces with, say, the usual topology, Hilbert and Banach spaces, etc. Can do multi-dimensional Markov processes, and much more. And you can have each point on the real line an event. Fine. But you just can't take the uncountable union of any set of such events and assume that the result is also an event. As for the event a random variable takes a value >= 0? Fine. Or, let the Borel subsets of the real line be the smallest sigma algebra that contains all the open sets, e.g., all the open intervals. Then for Borel set A and real valued random variable X, can ask for the probability X is in A. I believe you will find that you will have a solid foundation for what you want. To see all this stuff, need more than just the sparse definitions and, instead, need an actual text and maybe a course. Recently looked at the on-line materials from MIT and didn't see such a course. Graduate probability is not all that popular in the US; stochastic processes in continuous time is still less popular. To study graduate probability, I'd recommend a good undergraduate major in pure math with good coverage of, say, W. Rudin, Principles of Mathematical Analysis. Then good coverage of linear algebra from more than one of the best known texts. Likely also spend as much time as you can in Halmos, Finite Dimensional Vector Spaces. E.g., at one time, Halmos, Rudin, and Spivak, Calculus on Manifolds were the three main texts for Harvard's famous Math 55. Get good with proving the theorems. I also recommend Fleming, Functions of Several Variables. Then, sure, Royden, Real Analysis. Couldn't be prettier. If not in a hurry, then the real half of Rudin's Real and Complex Analysis. Especially if you like Fourier theory! Then of the probability books, I believe that the nicest, first book is L. Breiman, Probability. He wrote that before he went consulting and came back and did CART and random forests. Next, K. Chung, A Course in Probability Theory. Next, J. Neveu, Mathematical Foundations of the Calculus of Probability. Then, Loeve, Probability Theory. Loeve is huge -- mostly just use it for reference or browse. E.g., it has sufficient statistics and stationary stochastic processes (the EEs love that) IIRC not in the other books. IIRC, both Breiman and Neveu were Loeve students at Berkeley. If do well with Breiman, then for graduate probability, likely can stop there. Else, Chung will then be fast and easy reading and reinforce what you learned in Breiman. Neveu is elegant; my favorite, but deserve extra credit for each workable exercise you can find, not actually work, you understand, just find! Sure, some of the exercises are terrific, half a course in a few lines of an exercise. E.g., he has one of those on statistical decision theory or some such. And see the Tulcea material in the back. Then there's more that you can do on stochastic processes, potential theory via Brownian motion, e.g., for mathematical finance, stochastic optimal control, and more.

nerd_stuff 11y ago

Out of curiosity, what's the background that lets you be so dismissive of a Fields Medal winner? At this point I see a blog post written by a well respected mathematician whom I feel comfortable trusting and it's being brushed aside by I don't know who.

jonnybgood 11y ago
Being a great mathematician doesn't necessarily make a great math educator. It is odd Tao chose Durret, but I assume it's due to the book being freely available online.
- nerd_stuff 11y ago
  As the blog comments suggest there's a difference between a good self-study book and a good textbook for a class. The book will be accompanied by (at least) what look to be a good set of course notes. Some googling around suggests his students are quite pleased with him. He has, according to Wikipedia's intro for him, the undisputed king of math blogs. Both of these point to him being at least a good or above average educator.
d9fb698e010974b 11y ago
Well you can just read the reviews to see that the Durrett text isn't well regarded while others like Chung's are. And the criticism about a probability space not being a sample space is correct, but I think it's clear what Tao meant there, namely that the sample space would be a part of a probability space.
- keithpeter 11y ago
  Thanks The Amazon UK reviews of Chung's book lead me to A Probability Path by Sidney Resnick which appears to be aimed at non-mathematicians. I have invested (speaking as a renegade physicist lacking a systematic exploration of measure theory).
copperx 11y ago
Are you advocating "proof by authority "?

deleted 11y ago

[deleted]

olalonde 11y ago

> In the standard foundations of probability theory, as laid out by Kolmogorov, we can then model these events and random variables by introducing a sample space (which will be a probability space) to capture all the ambient sources of randomness; events are then modeled as measurable subsets of this sample space, and random variables are modeled as measurable functions on this sample space. This matches the definitions of Wikipedia pretty closely: "A probability space consists of three parts: A sample space [...], A set of events [...], The assignment of probabilities to the events". So either you misunderstood that sentence or both Wikipedia and Terrence Tao are wrong.

d9fb698e010974b 11y ago
The sample space is not the probability space. The probability space has a sample space though, which is what the Wikipedia definition says, but not what Tao literally said. I think it is pretty clear what Tao meant though.
graycat 11y ago
" ... a sample space (which will be a probability space)" is, in a word, wrong. The sample space will be part of a probability space but will not "be a probability space".
- elliptic 11y ago
  Nonsense. Tao is using perfectly idiomatic language here - "The sample space will be a probability space (once we have endowed it with some additional structure)".
  - graycat 11y ago
    That's not good: We don't want to have to use words such as endowed and structure that are not precisely defined in the context and, thus, are conceptually fuzzy. Some intuitive overviews, clearly labeled as such, are fine and can be helpful, but "idiomatic language" just is not. Won't find such in the writings of W. Rudin, P. Halmos, J. Neveu, or any of a long list of authors of some of the best math books. In a good math or computer science journal, a reviewer or the editor would likely reject "idiomatic language". The definitions I gave are the accepted ones.

mturmon 11y ago

To each their own. Durrett's text is perfectly fine. The books you mention are older, and perhaps you originally learned from them, so you still hold a torch? In particular, sending people to Loeve for their first course in measure theoretic probability would be really cruel. Personally, I learned from Feller, and Billingsley, and finally Durrett.

graycat 11y ago
> To each their own. Durrett's text is perfectly fine. The books you mention are older, and perhaps you originally learned from them, so you still hold a torch? Durrett seems on most of the topics to have less than any of the four texts I mentioned. > In particular, sending people to Loeve for their first course in measure theoretic probability would be really cruel. I listed the authors in reverse order in which to read them! The easiest start is the last, Breiman. But some things in Loeve are good, e.g., sufficient statistics. Neveu is my favorite as the most elegant, but his exercises are the most difficult. Breiman and Chung are fine. IIRC, both Breiman and Neveu were Loeve students. > Personally, I learned from Feller, and Billingsley, and finally Durrett. I used Feller I and/or II for reference. Otherwise his writing seemed to lack an overall unification. The only Billingsley text I used, and then only for a small topic, was Convergence of Probability Measures.