3 ms·

Ok, this looks awesome. I've been meaning to get some use from deep learning. However. . . I'd like to read about a bit more specifics as to what exactly in da

Ok, this looks awesome. I've been meaning to get some use from deep learning. However. . . I'd like to read about a bit more specifics as to what exactly in daytrading or stock trading in general using deep networks can figure out particularly. If you can provide something more than just saying it does predict prices and gives reasons as to "why" it is accurate each and every step I will definitely use this. The issue is generally the explanation for this article is too darn long. Split it up. There is at least 2 different parts here. One is getting the data ready for training including what you are looking at and why. The second is doing some of the training and analysing what needs to be changed. And maybe a third which would be some analysis of what should be changed. If you create a forum like the one from quantopian.com where people can share/discuss their models then this is a no brainer for use.

dave_sullivan 13y ago
The post did come about a bit long... I considered splitting into pieces, but I also wanted the first part to have some pretty specific results in it. Re: specifics on what it's learning: The short answer is no one knows. It's actually an active area of research, and while there are some methods for vision problems to visualize the sensitivity of feature detectors (they end up looking like faces, for example), it's harder to make sense of other types of data. In general, an RNN is learning 2 things: 1) a function to transform the data at a static level (so, in this case, one day of trading) and 2) a function to transform that learned representation/state across time in a way that reflects the dynamics found in the data. One of the reasons I built ersatz is because I wanted more rapid experiment results. Now that I've got that, I'm just beginning to scratch the surface on what's possible. Now that the machinery is in place, we're building out better tools for analyzing the data and what the predictions have to do with it. I share your general concerns re: all of that--which is why I've decided to stick to startups instead of day trading myself. Think of this post as more of a toy example/demonstration, albeit a promising one. Re: a forum--yeah, that's a good idea, we will do that.
- switch33 13y ago
  Thanks for the reply. From what I've read researching deep learning is still very promising regardless if it's a new field or not. I was just wondering if you had any idea of how to interpret results a bit more. It seems that I'll have to find more reading on this stuff though. I imagine deep learning to be more predictive than simple regression testing and some of the back propagation type predictors but I can't imagine it being extremely predictive when the stockmarket is known for being very subjective to attitudes as well as other unpredictable factors. Deep learning is supposed to pick out factors that may provide some information about things. As another test I'd recommend trying to do deep learning with weather data and a few stocks. This might give you something more discerning because the whole point of deep learning is to identify unknown factors that effect things. People have long suggested that weather plays a good factor in certain stocks but it'd be interesting to see what a trained model thinks.

dave_sullivan 13y ago

The post did come about a bit long... I considered splitting into pieces, but I also wanted the first part to have some pretty specific results in it. Re: specifics on what it's learning: The short answer is no one knows. It's actually an active area of research, and while there are some methods for vision problems to visualize the sensitivity of feature detectors (they end up looking like faces, for example), it's harder to make sense of other types of data. In general, an RNN is learning 2 things: 1) a function to transform the data at a static level (so, in this case, one day of trading) and 2) a function to transform that learned representation/state across time in a way that reflects the dynamics found in the data. One of the reasons I built ersatz is because I wanted more rapid experiment results. Now that I've got that, I'm just beginning to scratch the surface on what's possible. Now that the machinery is in place, we're building out better tools for analyzing the data and what the predictions have to do with it. I share your general concerns re: all of that--which is why I've decided to stick to startups instead of day trading myself. Think of this post as more of a toy example/demonstration, albeit a promising one. Re: a forum--yeah, that's a good idea, we will do that.

switch33 13y ago
Thanks for the reply. From what I've read researching deep learning is still very promising regardless if it's a new field or not. I was just wondering if you had any idea of how to interpret results a bit more. It seems that I'll have to find more reading on this stuff though. I imagine deep learning to be more predictive than simple regression testing and some of the back propagation type predictors but I can't imagine it being extremely predictive when the stockmarket is known for being very subjective to attitudes as well as other unpredictable factors. Deep learning is supposed to pick out factors that may provide some information about things. As another test I'd recommend trying to do deep learning with weather data and a few stocks. This might give you something more discerning because the whole point of deep learning is to identify unknown factors that effect things. People have long suggested that weather plays a good factor in certain stocks but it'd be interesting to see what a trained model thinks.