The only firm that I'm aware of that actually trades on twitter data is Derwent Capital Management. Strangely for a company that claims to be sitting on a cryst

The only firm that I'm aware of that actually trades on twitter data is Derwent Capital Management. Strangely for a company that claims to be sitting on a crystal ball for the markets they've chosen to provide a platform for others to trade on rather than simply making all those billions themselves, odd that. I'm well aware of the concept of the wisdom of the crowd but if your incoming data is noise then your ability to build any kind of aggregate analysis on top of it is going to be nil & Post-hoc analysis means you can find only things you already know are there. ANEW/AFINN etcetc are basically the white flag to any kind of meaningful automated analysis of tweets and resorting to simply dumb word counts instead. Yes, you can capture a broad pattern but the shape and strength of that? The contours are an artifact of the list you use, it's utterly arbitrary. Throw a few more random phrases on the list, assign them some arbitrary values and presto, new results! If an tweet goes round twitter in a minute with hundreds of thousands of RTs this kind of analysis will miss it completely unless it's lucky enough to be preloaded with the right dictionary. To work in this kind of context an algorithm has to be able to trim its own sails. There's nothing particularly wrong with your work, I'm just sick of the cycles wasted attempting to do an extreme version of the problems that NLP already struggles with.