3 ms·

You can do it; you just need to only evaluate your algorithm on data it hasn't been trained on. The same as with any machine learning problem really. Though it

You can do it; you just need to only evaluate your algorithm on data it hasn't been trained on. The same as with any machine learning problem really. Though it is indeed dismaying how many programmers dabbling into ML tend to do so without any scientific rigor... Whether the stockmarket can or cannot be predicted on the short term based on its past is another question... but I've been gathering some convincing evidence that it cannot (as in, its variations have no intrinsic structure; though it can still be predicted based on various external factors).

fawce 14y ago
what's an example of an external factor? maybe I can find it on quandl and backtest it.
- fchollet 14y ago
  The stockmarket is primarily driven by the news, both on the ultra-short term (minute scale) and at the scale of a few days. So, scraping the web and performing semantic analysis of investor sentiment can be a good way to get an edge in the game. Though in order to be really effective you'd need to do it before Wall Street traders' reactions adjust the prices, ie. get access to Bloomberg's B-Pipe [1], do real time semantic analysis and place orders with ultra low latency. Which quite a few trading firms are already doing... Other factors include P/E ratio (only accurate in the longer term), div yield, recent growth... [1] http://www.bloomberg.com/enterprise/enterprise_products/data_optimization/data_feeds/ http://www.bloomberg.com/enterprise/enterprise_products/data...
Homunculiheaded 14y ago
Cross validation doesn't change the fact that you're trying to predict a non-stationary distribution. Machine learning techniques generally make an assumption that your samples are being drawn from at least a close enough distribution to any future data that may enter the system. With problems like text classification in NLP you can generally make a safe assumption that, though language does change, it changes slowly enough that you don't have to worry. In other cases a model may simply need to be retrained after a certain period of time. Even in text classification real world systems will often incorporate an unsupervised novelty detector as well to indicate when the models may need to be retrained. Additionally normal cross validation (random or stratified sampling) does not work on time series data since you are in effect cheating by training on events in the future, which your model will not be able to predict. In order to test your model on time series data you would need to hold out a significant chunk of data at the end of the time series. If you're interested in the general case of predicting the stock market you might enjoy reading up on the Efficient Market Hypothesis[0] which deals quite well with just that question. http://en.wikipedia.org/wiki/Efficient-market_hypothesis http://en.wikipedia.org/wiki/Efficient-market_hypothesis
- fchollet 14y ago
  Indeed. One reason why ML is quite powerless when looking at the stockmarket (from the point of view of past stock values) is that, even if you assume there is a latent model to be learned, it would seem reasonable that this model evolves as fast as the economic landscape and that is to say, quite fast. To such an extent that the valid data points available (the past few years of price variations) would not suffice to train even a very low-dimensional latent model.

fawce 14y ago

what's an example of an external factor? maybe I can find it on quandl and backtest it.

fchollet 14y ago
The stockmarket is primarily driven by the news, both on the ultra-short term (minute scale) and at the scale of a few days. So, scraping the web and performing semantic analysis of investor sentiment can be a good way to get an edge in the game. Though in order to be really effective you'd need to do it before Wall Street traders' reactions adjust the prices, ie. get access to Bloomberg's B-Pipe [1], do real time semantic analysis and place orders with ultra low latency. Which quite a few trading firms are already doing... Other factors include P/E ratio (only accurate in the longer term), div yield, recent growth... [1] http://www.bloomberg.com/enterprise/enterprise_products/data_optimization/data_feeds/ http://www.bloomberg.com/enterprise/enterprise_products/data...

Homunculiheaded 14y ago

Cross validation doesn't change the fact that you're trying to predict a non-stationary distribution. Machine learning techniques generally make an assumption that your samples are being drawn from at least a close enough distribution to any future data that may enter the system. With problems like text classification in NLP you can generally make a safe assumption that, though language does change, it changes slowly enough that you don't have to worry. In other cases a model may simply need to be retrained after a certain period of time. Even in text classification real world systems will often incorporate an unsupervised novelty detector as well to indicate when the models may need to be retrained. Additionally normal cross validation (random or stratified sampling) does not work on time series data since you are in effect cheating by training on events in the future, which your model will not be able to predict. In order to test your model on time series data you would need to hold out a significant chunk of data at the end of the time series. If you're interested in the general case of predicting the stock market you might enjoy reading up on the Efficient Market Hypothesis[0] which deals quite well with just that question. http://en.wikipedia.org/wiki/Efficient-market_hypothesis http://en.wikipedia.org/wiki/Efficient-market_hypothesis

fchollet 14y ago
Indeed. One reason why ML is quite powerless when looking at the stockmarket (from the point of view of past stock values) is that, even if you assume there is a latent model to be learned, it would seem reasonable that this model evolves as fast as the economic landscape and that is to say, quite fast. To such an extent that the valid data points available (the past few years of price variations) would not suffice to train even a very low-dimensional latent model.