4 ms·

Random Variable Algebra

47 points by wvlia5 5y ago

foxes 5y ago
I struggle with the notation, where functions are defined, and if things should be lifts/pullbacks/pushforwards. X is a real random variable. So it is a map from some space of measurements, M , X : M -> R. Z is supposed to be a random variable in this case (why lower case?). f is a function from R->R, and you mean the pullback to Omega? Clearly all these spaces at least you can have some sensible notion of differential forms, differentials, so I imagine a space \Omega_M/R, so there are derivations, generalised stokes theorem etc. Then even though this is all some analytic kind of world, well we can get the algebraic structure right at least. If things are supposed to for example, Borel or whatever quantifier, I feel like that is just extra data and constraints. Stats overall seems kind of sloppy.
- wvlia5 5y ago
  f is a function from Rⁿ→R X is a multidimensional random variable, z is unidimensional, so that's why the difference in casing
- whatshisface 5y ago
  In the language of differential forms, a probability distribution would be a form of the highest order (dx ^ dy ^ dz in three dimensions), and f would be your typical map between manifolds. It would then receive identical treatment to a density, because in a sense that's what PDFs are - densities of samples in the limit of infinite samples. It is hardly sloppy, the author just wrote it in the language of vector calculus. Another treatment, the one in my textbooks, addresses it with the determinant of the Jacobian matrix from the perspective of change-of-coordinates in multidimensional integrals. As always there are as many ways to skin a cat as there are closed curves on the 2-sphere.
  - r-zip 5y ago
    It is very sloppy when you come from say, analysis, where things are precisely defined, especially in more applied statistics. In my experience, any more applied than probability theory itself and you get omissions of very important details such as the probability space under consideration and the RVs involved in expectations, in addition to frequent inconsistencies and outright abuses of notation. These tendencies make it a harder subject for someone who is used to precise exposition to pick up.
pizza 5y ago
Nice article. Could you make like a picture to show this? Would be nice for intuition.
- wvlia5 5y ago
  I will. A picture of what would be useful?
  - pizza 5y ago
    Not sure, to be honest. I guess anything you feel that displays the machinations of your proof.
hwita 5y ago
I feel like more explanation is needed between steps 4 and 5 of the proof.
- wvlia5 5y ago
  Visualize it. You are integrating a function P(X) along a thin surface defined by z<f(X)<z+dz. What is volume dV for a given dz? It is a surface dS times dX: the amount that X changes for a given dz. Since z=f(x), then dx=dz/|∇f(x)|. So dV=dS*dz/|∇f(x)|
fabiom 5y ago
Interesting. But by the title I thought it was about an algebra (or algebraic structure) of random variables.
- wvlia5 5y ago
  It presents how every F defined on random variables maps to a F' defined on the probability distribution of such random variables, an isomorphism.
  - sasaf5 5y ago
    I think parent was expecting to see things like operations between random variables and their typing system. I am trying to imagine how all of this could be derived from the isomorphism presented, but it's beyond my mathematical reach.
quenix 5y ago
Somewhat unrelated, but as a Notion lover and engineer I was so relieved when the team added native inline LaTeX support to the product. Gone are the days where we had to contend with ugly Unicode characters or italicised Times New Roman as an alternative. This document looks superb.
jpfr 5y ago
Uhm. That's not exactly a new result. See Theorem 2.7 (page 34) in [1] for the classical formulation. If the advantage is in the notation, please enlighten me. [1] Jazwinski, Andrew H. Stochastic Processes and Filtering Theory. Academic Press, 1970. https://books.google.fr/books?id=4AqL3vE2J-sC&pg=PA34 https://books.google.fr/books?id=4AqL3vE2J-sC&pg=PA34
- derbOac 5y ago
  FWIW, 2.85 in [1] is not obviously the same to me as linked statement. Not saying it isn't the same but seems like there's some steps there. I agree it seems like something I've seen before though.
anon_tor_12345 5y ago
this is literally just a change of variable https://en.wikipedia.org/wiki/Probability_density_function#Function_of_random_variables_and_change_of_variables_in_the_probability_density_function https://en.wikipedia.org/wiki/Probability_density_function#F... every single probability/stats book covers it in the first chapter.
HiddenSmirk 5y ago
Somewhat unrelated, but as a Notion lover and engineer I was so relieved when the team added native inline LaTeX support to the product. Gone are the days where we had to contend with ugly Unicode characters or italicised Times New Roman as an alternative. https://www.greatpeople-me.org/ https://www.greatpeople-me.org/

47 points by wvlia5 5y ago

I struggle with the notation, where functions are defined, and if things should be lifts/pullbacks/pushforwards. X is a real random variable. So it is a map from some space of measurements, M , X : M -> R. Z is supposed to be a random variable in this case (why lower case?). f is a function from R->R, and you mean the pullback to Omega? Clearly all these spaces at least you can have some sensible notion of differential forms, differentials, so I imagine a space \Omega_M/R, so there are derivations, generalised stokes theorem etc. Then even though this is all some analytic kind of world, well we can get the algebraic structure right at least. If things are supposed to for example, Borel or whatever quantifier, I feel like that is just extra data and constraints. Stats overall seems kind of sloppy.

wvlia5 5y ago
f is a function from Rⁿ→R X is a multidimensional random variable, z is unidimensional, so that's why the difference in casing
whatshisface 5y ago
In the language of differential forms, a probability distribution would be a form of the highest order (dx ^ dy ^ dz in three dimensions), and f would be your typical map between manifolds. It would then receive identical treatment to a density, because in a sense that's what PDFs are - densities of samples in the limit of infinite samples. It is hardly sloppy, the author just wrote it in the language of vector calculus. Another treatment, the one in my textbooks, addresses it with the determinant of the Jacobian matrix from the perspective of change-of-coordinates in multidimensional integrals. As always there are as many ways to skin a cat as there are closed curves on the 2-sphere.
- r-zip 5y ago
  It is very sloppy when you come from say, analysis, where things are precisely defined, especially in more applied statistics. In my experience, any more applied than probability theory itself and you get omissions of very important details such as the probability space under consideration and the RVs involved in expectations, in addition to frequent inconsistencies and outright abuses of notation. These tendencies make it a harder subject for someone who is used to precise exposition to pick up.

pizza 5y ago

Nice article. Could you make like a picture to show this? Would be nice for intuition.

wvlia5 5y ago
I will. A picture of what would be useful?
- pizza 5y ago
  Not sure, to be honest. I guess anything you feel that displays the machinations of your proof.

hwita 5y ago

I feel like more explanation is needed between steps 4 and 5 of the proof.

wvlia5 5y ago
Visualize it. You are integrating a function P(X) along a thin surface defined by z<f(X)<z+dz. What is volume dV for a given dz? It is a surface dS times dX: the amount that X changes for a given dz. Since z=f(x), then dx=dz/|∇f(x)|. So dV=dS*dz/|∇f(x)|

fabiom 5y ago

Interesting. But by the title I thought it was about an algebra (or algebraic structure) of random variables.

wvlia5 5y ago
It presents how every F defined on random variables maps to a F' defined on the probability distribution of such random variables, an isomorphism.
- sasaf5 5y ago
  I think parent was expecting to see things like operations between random variables and their typing system. I am trying to imagine how all of this could be derived from the isomorphism presented, but it's beyond my mathematical reach.

quenix 5y ago

Somewhat unrelated, but as a Notion lover and engineer I was so relieved when the team added native inline LaTeX support to the product. Gone are the days where we had to contend with ugly Unicode characters or italicised Times New Roman as an alternative. This document looks superb.

jpfr 5y ago

Uhm. That's not exactly a new result. See Theorem 2.7 (page 34) in [1] for the classical formulation. If the advantage is in the notation, please enlighten me. [1] Jazwinski, Andrew H. Stochastic Processes and Filtering Theory. Academic Press, 1970. https://books.google.fr/books?id=4AqL3vE2J-sC&pg=PA34 https://books.google.fr/books?id=4AqL3vE2J-sC&pg=PA34

derbOac 5y ago
FWIW, 2.85 in [1] is not obviously the same to me as linked statement. Not saying it isn't the same but seems like there's some steps there. I agree it seems like something I've seen before though.

anon_tor_12345 5y ago

this is literally just a change of variable https://en.wikipedia.org/wiki/Probability_density_function#Function_of_random_variables_and_change_of_variables_in_the_probability_density_function https://en.wikipedia.org/wiki/Probability_density_function#F... every single probability/stats book covers it in the first chapter.

HiddenSmirk 5y ago