5 ms·

What kind of operation happens when you pointwise multiply two probability density functions? It wouldn't even integrate to 1.0 any more (it can even integrate

What kind of operation happens when you pointwise multiply two probability density functions? It wouldn't even integrate to 1.0 any more (it can even integrate to 0.0, like in the example 4). Rather than taking the convolution, which corresponds to adding two independent random variables that generate said density functions.

Mithriil 6y ago
Pointwise multiplication of, say, two density functions gives the likelihood function of two independent random variables with these densities. This function is not a density function, as you say, but is still an important object, as it is used for inference based on the likelihood principle.
- H8crilA 6y ago
  I'm sorry, but I'm lost, are we talking about this likelihood function? I'm guessing no, since that's an object defined in a statistical space, not in a probabilistic space: https://en.m.wikipedia.org/wiki/Likelihood_function https://en.m.wikipedia.org/wiki/Likelihood_function
  - lalaithion 6y ago
    Oftentimes, a “likelihood function” is generalized from that definition to refer to any non-normalized function which is still treated like a probability distribution. Any function with a finite integral over the real line can be turned into a probability distribution easily, and there are other functions that can be used in places where you would normally use a probability function, even when they don’t integrate to a finite value over the real line. (For example, improper priors in Bayesian analysis.) “Function that can in some contexts be treated like a probability distribution” is just too long.
  - moultano 6y ago
    Yes, that's the right likelihood function. To get probabilities distributions out of the products in the post, all you have to do is normalize them. That would be equivalent to applying bayes rule and combining independent evidence about a parameter with an uninformative prior. Normalizing doesn't change the shapes, so I left out the details.
- srean 6y ago
  But then the term would be P(x1) P(x2) and the correct summation/integration to use would be over the cartesian product of the sample space. Under such summation/integration its a valid density
epistasis 6y ago
Joint probability of independent events, especially the common case of independent, identically distributed data. (Or exchangeable observations, in the Bayesian framework). For example, flipping a coin n times, observing heights of people in a population, etc.
- H8crilA 6y ago
  But we're talking about "continuous variables", random variables with infinite support for which P(X=a)=0, for all a. It's kind of like just a single point in the convolution for "X_1 - X_2", corresponding to "X_1 - X_2 = 0" ?
  - epistasis 6y ago
    I'm not sure I understand what you are trying to convey. If we have two continuous random variables in a probability space, we have a mapping P(X, Y): R^2 --> R, such that the integral sums to 1. If X and Y are independent, then P(X, Y) == P(X)P(Y). The examples in the linked article are a bit confusing because some of them have P(X) and P(Y) as different distributions. But the plot only shows the plot of the 2D domain along the 1D line of X==Y, which isn't a terribly common occurrence in actual practice.
    - H8crilA 6y ago
      P(X,Y), when integrated along all possible diagonals given by x+y=z where z goes through all real numbers, is what a convolution is (of P(X) and P(Y) generated by those variables). This gives you the distribution of the variable Z=X+Y. So, we're talking about the same thing, namely that multiplying two probability density functions pointwise roughly corresponds to a single point where X==Y (the same as X-Y==0), but otherwise doesn't seem terribly useful.
Enginerrrd 6y ago
Wouldn't you multiply like you do for functions as an inner product for hilbert space? Something like: $ f \cdot g = \int_\infty^\infty f(x)g(x)dx $ No? The schwartz inequality gives us that the product then has a value between 0 and 1. ...Not sure what it means if the value is less than one. What does orthogonality mean when it comes to probability density functions?
- sn41 6y ago
  Usually orthogonal means uncorrelated random variables (not necessarily independent). A good example is sine and cosine - they are uncorrelated in the interval 0 to pi. But clearly they are not independent.
  - Mithriil 6y ago
    Sine and cosine can be negative. Density functions cannot, and thus, orthogonality of the density functions implies that they don't have the same support with probability 1, or, said otherwise, the measure of the intersection of their support is 0.
- andi999 6y ago
  Scalar product doesnt give you a function/density whatever, just a number (which can be important, but this doesnt adress the question)
adenadel 6y ago
You might be interested in the idea of a copula and Sklar's theorem https://en.wikipedia.org/wiki/Copula_(probability_theory)#Sklar's_theorem https://en.wikipedia.org/wiki/Copula_(probability_theory)#Sk...

Mithriil 6y ago

Pointwise multiplication of, say, two density functions gives the likelihood function of two independent random variables with these densities. This function is not a density function, as you say, but is still an important object, as it is used for inference based on the likelihood principle.

H8crilA 6y ago
I'm sorry, but I'm lost, are we talking about this likelihood function? I'm guessing no, since that's an object defined in a statistical space, not in a probabilistic space: https://en.m.wikipedia.org/wiki/Likelihood_function https://en.m.wikipedia.org/wiki/Likelihood_function
- lalaithion 6y ago
  Oftentimes, a “likelihood function” is generalized from that definition to refer to any non-normalized function which is still treated like a probability distribution. Any function with a finite integral over the real line can be turned into a probability distribution easily, and there are other functions that can be used in places where you would normally use a probability function, even when they don’t integrate to a finite value over the real line. (For example, improper priors in Bayesian analysis.) “Function that can in some contexts be treated like a probability distribution” is just too long.
- moultano 6y ago
  Yes, that's the right likelihood function. To get probabilities distributions out of the products in the post, all you have to do is normalize them. That would be equivalent to applying bayes rule and combining independent evidence about a parameter with an uninformative prior. Normalizing doesn't change the shapes, so I left out the details.
srean 6y ago
But then the term would be P(x1) P(x2) and the correct summation/integration to use would be over the cartesian product of the sample space. Under such summation/integration its a valid density

epistasis 6y ago

Joint probability of independent events, especially the common case of independent, identically distributed data. (Or exchangeable observations, in the Bayesian framework). For example, flipping a coin n times, observing heights of people in a population, etc.

H8crilA 6y ago
But we're talking about "continuous variables", random variables with infinite support for which P(X=a)=0, for all a. It's kind of like just a single point in the convolution for "X_1 - X_2", corresponding to "X_1 - X_2 = 0" ?
- epistasis 6y ago
  I'm not sure I understand what you are trying to convey. If we have two continuous random variables in a probability space, we have a mapping P(X, Y): R^2 --> R, such that the integral sums to 1. If X and Y are independent, then P(X, Y) == P(X)P(Y). The examples in the linked article are a bit confusing because some of them have P(X) and P(Y) as different distributions. But the plot only shows the plot of the 2D domain along the 1D line of X==Y, which isn't a terribly common occurrence in actual practice.
  - H8crilA 6y ago
    P(X,Y), when integrated along all possible diagonals given by x+y=z where z goes through all real numbers, is what a convolution is (of P(X) and P(Y) generated by those variables). This gives you the distribution of the variable Z=X+Y. So, we're talking about the same thing, namely that multiplying two probability density functions pointwise roughly corresponds to a single point where X==Y (the same as X-Y==0), but otherwise doesn't seem terribly useful.

Enginerrrd 6y ago

Wouldn't you multiply like you do for functions as an inner product for hilbert space? Something like: $ f \cdot g = \int_\infty^\infty f(x)g(x)dx $ No? The schwartz inequality gives us that the product then has a value between 0 and 1. ...Not sure what it means if the value is less than one. What does orthogonality mean when it comes to probability density functions?

sn41 6y ago
Usually orthogonal means uncorrelated random variables (not necessarily independent). A good example is sine and cosine - they are uncorrelated in the interval 0 to pi. But clearly they are not independent.
- Mithriil 6y ago
  Sine and cosine can be negative. Density functions cannot, and thus, orthogonality of the density functions implies that they don't have the same support with probability 1, or, said otherwise, the measure of the intersection of their support is 0.
andi999 6y ago
Scalar product doesnt give you a function/density whatever, just a number (which can be important, but this doesnt adress the question)

adenadel 6y ago

You might be interested in the idea of a copula and Sklar's theorem https://en.wikipedia.org/wiki/Copula_(probability_theory)#Sklar's_theorem https://en.wikipedia.org/wiki/Copula_(probability_theory)#Sk...