4 ms·
From the top right of page 6: To assess the statistical significance of the SOFNN achieving the above mentioned accuracy of 87.6% in predicting the up
by Robin_Message 16y ago
From the top right of page 6:
To assess the statistical significance of the SOFNN achieving
the above mentioned accuracy of 87.6% in predicting the up and down
movement of the DJIA we calculate the odds of this result occurring
by chance. The binomial distribution in- dicates that the probability
of achieving exactly 87.6% correct guesses over 15 trials (20 days
minus weekends) with a 50% chance of success on each single trial
equals 0.32%. Taken over the entire length of our data set (February
28 to December 20, excluding weekends) we find approximately 10.9
of such 20 day periods. The odds that the mentioned probability
would hold by chance for a random period of 20 days within that
period is then estimated to be 1−(1−0.0032)10.9 = 0.0343 or
3.4%. The SOFNN direction accuracy is thus most likely not the result
of chance nor our selecting a specifically favorable test period.
I'm not sure about the result not being chance. In particular, aren't they privileging the hypothesis by using the probability of 87.6% guesses being correct? They say the result only had a 3.4% chance of happening, yet we could assign a similar probability to a wide variety of other seemingly unlikely outcomes. I'd much rather have seen the predictor tuned on a subset of the data and then tested on the rest of the data - perhaps the authors will release their predictor and we can try it on 2009.
Given that twitter and the stock market are both correlated to what is happening in the world (in fact, caused by what is happening in the world,) it is not surprising that they are correlated with each other. Somehow I don't think correlation in the other direction would get the upvotes though - stock market crash predicts sadness on twitter? You don't say.
In short, I don't follow all of their statistics, but I am highly dubious of their model having any predictive power.
- b0b0b0b 16y agoThey managed to test and train on different periods of time. February 28, 2008 to November 28, 2008 is chosen as the longest possible training period while Dec 1 to Dec 19, 2008 was chosen as the test period That said, their split exhibits enormous bias: Dec 1 to Dec 19, 2008 was chosen as the test period because it was characterized by stabilization of DJIA values after considerable volatility in previous months and the absence of any unusual or significant socio cultural events I don't understand why they would think this is okay.
- Robin_Message 16y agoWhoops, missed that. Well then, in the absence of unpredictable events, twitter could predict the stock market, but it can't anymore since the market will now factor this new knowledge in (assuming perfect markets of course.)
- enjo 16y agoPendantic response: Perfectly efficient markets. Which, as a theory, has been under rather serious siege for the better part of the last decade. You can certainly make a behavioral argument that would reach more or less the same conclusion, however.
- enjo 16y agoPendantic response: Perfectly efficient markets. Which, as a theory, has been under rather serious siege for the better part of the last decade. You can certainly make a behavioral argument that would reach more or less the same conclusion, however.
- pessimizer 16y agoI can see exactly why they would think it was okay. What they were trying to do was to figure out whether twitter moods could predict the unpredictable random movements of the market, rather than the outside edges of the bell curve when anyone could have predicted what would happen, and where the market moves might even be seen as a cause of the twitter moods rather than a result, or a co-result of some other process. Basically, it's the hardest place to test the correlation. Since they got great results, it'll be interesting to see them extend the analysis and see if any interesting dynamics show up.
- pbhjpbhj 16y ago>perhaps the authors will release their predictor and we can try it on 2009 Can't we somehow look and see if the author's lifestyle suddenly became more lavish. That should be a good predictor of stock market success, but then ...