2 ms·

I have done countless LDA on tweets and it worked great. here is an example of taking tweets that have 'diabetes' and correlating topics with counties. https:/

by autokad 10y ago

I have done countless LDA on tweets and it worked great. here is an example of taking tweets that have 'diabetes' and correlating topics with counties. https://www.linkedin.com/in/karl-dailey-02557b65/treasury/position:755225215/?entityUrn=urn%3Ali%3Afs_treasuryMedia%3A(ACoAAA3JzV0BT9zF9Db_PxdLfE28gQQfK_PzmhM%2C1486424963566) https://www.linkedin.com/in/karl-dailey-02557b65/treasury/po... The original topics were actually better, but I was asked to adjust the language to wipe out common language across all tweets. You can still see interesting things going on though. North East: Charity, Hospitals, Research. South: Koolaid, sweat tea. etc. I had also done topic modeling on customer surveys for Comcast (I cant show them), but the topics identified 3 key features that lead to low customer satisfaction. I have also used LDA on grocery purchases (for mere fun), and it worked out really great as well.

stuartaxelowen 10y ago
But what real insights do the topics give? The only LDA results I've seen are "fairly obvious" or "not understandable". Seeing a "car, road, light, drive, trip, ..." topic is not insightful - this is a topic that is obvious to most humans. A more interesting topic would be one that is understandable but was not obvious, even to the people intimately involved in data's subject matter. These are harder to discover, but they do exist - and I have never seen LDA surface them.
- autokad 10y ago
  I see quantifiable and visualizible insights. Its also good for generating features, and I can also create interaction features with important n-grams. If it cant give you what you are looking for, maybe you are asking a fish to climb a tree
- nl 10y ago
  The purpose of LDA/topic visualisations isn't to do something humans can't do. It's to do it extremely rapidly on large amounts of data.

by autokad 10y ago

stuartaxelowen 10y ago

But what real insights do the topics give? The only LDA results I've seen are "fairly obvious" or "not understandable". Seeing a "car, road, light, drive, trip, ..." topic is not insightful - this is a topic that is obvious to most humans. A more interesting topic would be one that is understandable but was not obvious, even to the people intimately involved in data's subject matter. These are harder to discover, but they do exist - and I have never seen LDA surface them.

autokad 10y ago
I see quantifiable and visualizible insights. Its also good for generating features, and I can also create interaction features with important n-grams. If it cant give you what you are looking for, maybe you are asking a fish to climb a tree
nl 10y ago
The purpose of LDA/topic visualisations isn't to do something humans can't do. It's to do it extremely rapidly on large amounts of data.