Oooooh man, Latent Dirichlet Allocation is cool stuff, especially in the context of topic modeling (which is what Chen is doing here). OP actually wrote a pret

Oooooh man, Latent Dirichlet Allocation is cool stuff, especially in the context of topic modeling (which is what Chen is doing here). OP actually wrote a pretty accessible blog post about how he does this sort of thing which you can see at [1]. If you don't want to read it, Mike Jordan has a pretty neat presentation about it at [2]. If you're statistically trained and don't want to view the video, you will probably understand this synopsis: is that you can view each document in a set of documents as a discrete admixture of some number of topics. If you imagine that words can be modeled with a discrete exchangeable random variable, and you choose some number of topics to model (let's say k topics), then you can use a hierarchical Bayesian model, specifically with an underlying Dirichlet distribution over the base measure of each Dirichlet process that forms every particular admixture document. This allows topics to share some amount of information, which then allows you to generate some pretty useful topics, like the ones from the OP. If you don't understand that synopsis, then go look at Jordan's talk. It makes it all pretty clear. :) [1] http://blog.echen.me/2011/08/22/introduction-to-latent-dirichlet-allocation/ http://blog.echen.me/2011/08/22/introduction-to-latent-diric... [2] http://videolectures.net/icml05_jordan_dpcrp/ http://videolectures.net/icml05_jordan_dpcrp/