4 ms·
Not one mention of the EM algorithm, which is, as far as I can understand, is being described here (https://en.m.wikipedia.org/wiki/Expectation%E2%80%93maximiza
by Jun8 2y ago
Not one mention of the EM algorithm, which is, as far as I can understand, is being described here (https://en.m.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm https://en.m.wikipedia.org/wiki/Expectation%E2%80%93maximiza...). It has so many applications, among which is estimating number of clusters for a Gaussian mixture model.
An ELI5 intro: https://abidlabs.github.io/EM-Algorithm/ https://abidlabs.github.io/EM-Algorithm/
- Sniffnoy 2y agoIt does not appear to be what's being described here? Could you perhaps expand on the equivalence between the two if it is?
- miki123211 2y ago> It has so many applications, among which is estimating number of clusters for a Gaussian mixture model Any sources for that? As far as I remember, EM is used to calculate actual cluster parameters (means, covariances etc), but I'm not aware of any usage to estimate what number of clusters works best. Source: I've implemented EM for GMMs for a college assignment once, but I'm a bit hazy on the details.
- fleischhauf 2y agoyou are right you still need the number of clusters
- BrokrnAlgorithm 2y agoI've been out of the loop for stats for a while, but is there a viable approach for estimating ex ante the number of clusters when creating a GMM? I can think if constructing ex post metrics, i.e using a grid and goodness of fit measurements, but these feel more like brute forcing it
- disgruntledphd2 2y agoUnsupervised learning is hard, and the pick K problem is probably the hardest part. For PCA or factor analysis, there's lots of ways but without some way of determining ground truth it's difficult to know if you've done a good job.
- lukego 2y agoIs the question fundamentally: what's the relative likelihood of each number or clusters? If so then estimating the marginal likelihood of each one and comparing them seems pretty reasonable? (I mean in the sense of Jaynes chapter 20.)
- CrazyStat 2y agoThere are Bayesian nonparametric methods that do this by putting a dirichlet process prior on the parameters of the mixture components. Both the prior specification and the computation (MCMC) are tricky, though.
- CrazyStat 2y agoEM can be used to impute data, but that would be single imputation. Multiple imputation as described here would not use EM since the goal is to get samples from a distribution of possible values for the missing data.
- wdkrnls 2y agoIn other words, EM makes more sense. All this imputation stuff seems to me more like an effort to keep using obsolete modeling techniques.
- CrazyStat 2y agoAbsolutely not. EM imputation (or single imputation in general) fails to account for the uncertainty in imputed data. You end up with artificially inflated confidence in your results (p-values too small, confidence/credible intervals too narrow, etc.). Multiple imputation is much better.