4 ms·

They dont need to be independent but they need to be jointly gaussian, but it is a very small edge case I guess

AstralStorm 7y ago
You can check if this holds by inspecting the covariance matrix. It can be degenerate or the variables may be correlated (thus not joint normal) and even independent uncorrelated gaussian ones need not be. https://en.wikipedia.org/wiki/Multivariate_normal_distribution#Two_normally_distributed_random_variables_need_not_be_jointly_bivariate_normal https://en.wikipedia.org/wiki/Multivariate_normal_distributi... It's best to check using the second definition from: https://en.wikipedia.org/wiki/Multivariate_normal_distribution#Equivalent_definitions https://en.wikipedia.org/wiki/Multivariate_normal_distributi... This results in the BHEP test: https://en.wikipedia.org/wiki/Multivariate_normal_distribution#Multivariate_normality_tests https://en.wikipedia.org/wiki/Multivariate_normal_distributi...
- kgwgk 7y ago
  > You can check if this holds by inspecting the covariance matrix. Just a clarification: if that’s somehow correct then it’s not about the 2x2 covariance matrix that gives the linear relationship between two variables. Because two variables can be correlated and the joint distribution be a multivariate Gaussian. And two variables could be uncorrelated and not be independent; the covariance matrix doesn’t contain much information about the joint distribution (unless joint normality is assumed).
  - AstralStorm 7y ago
    Correct, that's only useful to check for degeneracy. The test is much more certain. And it's bigger than 2x2, it depends on number of dimensions you're checking. 3 variables goes 3x3 etc. For fitting as shown (and GANs), it gets big really fast.
    - kgwgk 7y ago
      I don't understand what covariance matrix you're talking about. Let's say we have two variables and the joint distribution is given by the probability density function p(x,y) which can have any form as long as it's always >=0 and integrates to 1. The covariance matrix is a 2x2 matrix and contains only three real numbers (four, but the off-diagonal terms are necessarily equal). The covariance matrix (together with the vector of means) gives a complete description of the joint distribution only when the joint distribution is completely specified by its first two moments and all its higher moments are zero (i.e. it is a multivariate normal). The fact that the correlation between the variables is zero doesn't mean necessarily that the variables are independent when the joint distribution is not a multivariate normal. The covariance matrix alone cannot be used to tell whether the joint distribution is a multivariate normal or not. Every possible covariance matrix defines a multivariate normal but is also compatible with an infinity of other distributions.
em500 7y ago
You're right and the author's note is wrong. Joint Normality of a random vector implies component-wise Normality, but not the other way around, so it's a stronger condition.

You can check if this holds by inspecting the covariance matrix. It can be degenerate or the variables may be correlated (thus not joint normal) and even independent uncorrelated gaussian ones need not be. https://en.wikipedia.org/wiki/Multivariate_normal_distribution#Two_normally_distributed_random_variables_need_not_be_jointly_bivariate_normal https://en.wikipedia.org/wiki/Multivariate_normal_distributi... It's best to check using the second definition from: https://en.wikipedia.org/wiki/Multivariate_normal_distribution#Equivalent_definitions https://en.wikipedia.org/wiki/Multivariate_normal_distributi... This results in the BHEP test: https://en.wikipedia.org/wiki/Multivariate_normal_distribution#Multivariate_normality_tests https://en.wikipedia.org/wiki/Multivariate_normal_distributi...

kgwgk 7y ago
> You can check if this holds by inspecting the covariance matrix. Just a clarification: if that’s somehow correct then it’s not about the 2x2 covariance matrix that gives the linear relationship between two variables. Because two variables can be correlated and the joint distribution be a multivariate Gaussian. And two variables could be uncorrelated and not be independent; the covariance matrix doesn’t contain much information about the joint distribution (unless joint normality is assumed).
- AstralStorm 7y ago
  Correct, that's only useful to check for degeneracy. The test is much more certain. And it's bigger than 2x2, it depends on number of dimensions you're checking. 3 variables goes 3x3 etc. For fitting as shown (and GANs), it gets big really fast.
  - kgwgk 7y ago
    I don't understand what covariance matrix you're talking about. Let's say we have two variables and the joint distribution is given by the probability density function p(x,y) which can have any form as long as it's always >=0 and integrates to 1. The covariance matrix is a 2x2 matrix and contains only three real numbers (four, but the off-diagonal terms are necessarily equal). The covariance matrix (together with the vector of means) gives a complete description of the joint distribution only when the joint distribution is completely specified by its first two moments and all its higher moments are zero (i.e. it is a multivariate normal). The fact that the correlation between the variables is zero doesn't mean necessarily that the variables are independent when the joint distribution is not a multivariate normal. The covariance matrix alone cannot be used to tell whether the joint distribution is a multivariate normal or not. Every possible covariance matrix defines a multivariate normal but is also compatible with an infinity of other distributions.

em500 7y ago

You're right and the author's note is wrong. Joint Normality of a random vector implies component-wise Normality, but not the other way around, so it's a stronger condition.