Yes, that is one of the biggest challenges for any machine learning system to be effective. And the more specific the data set, the harder it is to collect (or

Yes, that is one of the biggest challenges for any machine learning system to be effective. And the more specific the data set, the harder it is to collect (or get access to) and probably the more valuable. Someone in this thread cites the Enron emails. Odds are if you're building system to analyze energy trading or (maybe) financial trading in general in American English, then that will work. The problem is that that's a small segment. The main way to collect this data is to effectively do a Trojan Horse by building something else in a complementary area that is useful that lets you start collecting the data. Once you have it, machine learning is useful. This is what Boomerang has done for email and what Google has done with Google Voice.