4 ms·

Financial Time Series Forecasting with Deep Learning: A Literature Review

102 points by Bostonian 7y ago

w1ntermute 7y ago
The most important factor for financial time series forecasting is undoubtedly access to clean data. This is what sets Renaissance apart. There’s no need for particularly sophisticated math - they’ve been doing it for 3 decades.
- craze3 7y ago
  Renaissance? What is that...?
  - apollopower 7y ago
    Renaissance Technologies: https://en.wikipedia.org/wiki/Renaissance_Technologies https://en.wikipedia.org/wiki/Renaissance_Technologies Founded by Jim Simons, the "Father" of quantitative research. The firm is famous for their significant year-over-year returns and notorious for only hiring PhDs from mathematics/physics/computer science. There is an interesting book on how Jim Simons created the company and built his team of academics from the ground up, "The Man Who Solved the Market"
    - craze3 7y ago
      Oh yes I remember reading this wiki before. I just didn't remember what the company was called. I even tried to research it by googling "renaissance time series prediction," and still couldn't find it. I don't understand why I was downvoted? I don't think there was enough context for anyone (who's not in the field) to know what you meant, and I thought it would be useful for people like myself to get clued in.
      - aloukissas 7y ago
        This is the world of HN. If your comments don't fit in the cool crowd, you'll be downvoted.
    - zkid18 7y ago
      Probably Edward Thorp is the "Father" of quantitative research.Since the late 1960s, Thorp has used his knowledge of probability and statistics in the stock market by discovering and exploiting a number of pricing anomalies in the securities markets, and he has made a significant fortune [1] [1] https://en.wikipedia.org/wiki/Edward_O._Thorp https://en.wikipedia.org/wiki/Edward_O._Thorp
- deleted 7y ago
  [deleted]
- ackbar03 7y ago
  Are you sure that's the case? I dunno what background your coming from but I did pretty rigorous statistics in college including time series and stochastic processes and all that. Time series are notoriously difficult to forecast. Arch, garch, starch and all that sound good in theory but are pretty crap in practice
- apollopower 7y ago
  If access to clean data was what set Renaissance apart, we'd most likely see other funds/firms do just as well or Renaissance's returns be much lower. Market data services such as Reuters/Morning Star provide incredibly clean data at great convenience for anything you can imagine, even including astronomical data. Its kind of like the Black-Scholes model. If it really was that simple and straightforward, everybody would be using it to make a ton of money.
  - bob_theslob646 7y ago
    >Market data services such as Reuters/Morning Star provide incredibly clean data at great convenience for anything you can imagine, even including astronomical data. Incredibly clean ,that's a very big stretch. Source: have worked with TR sirca db for almost a decade.
- throwawaymath 7y ago
  That's not what sets Renaissance apart. Please don't add to the baseless speculation that gets tossed around about RenTech so often. Are you aware of just how many trading firms have access to phenomenally clean data? It drives me nuts when I see words like "undoubtedly" thrown around so confidently this way. A new book comes out about Simons and Renaissance, it enters the financial zeitgeist for a little while, and now everyone is apparently an expert on the firm's differentiating competency. For what it's worth, what you're saying is contradicted by Nick Patterson. He did not say that Renaissance had access to clean data no one else did. What he said is that in the early days, they spent almost all their time cleaning the data. In any case, that's table stakes these days. All successful quant firms spend time sourcing exceptional data and ensuring it's as polished as possible.
  - throwlaplace 7y ago
    >It drives me nuts when I see words like "undoubtedly" thrown around so confidently this way I see this very often on Reddit and here; someone reads a story and then behaves like they're suddenly a prophet whose job it is to inform the rest of us. On Reddit (because the threads are much bigger) I see this kind of repetition promulgation in the same thread. Here it's typically across threads (as you've noticed). I'll never understand what people get out seeming like an authority figure in a completely anonymous forum.
- zkid18 7y ago
  Could you elaborate the term of "clean financial data" please?
thedudeabides5 7y ago
Seems like a pretty big list of prior papers, which is a nice reference, but some notion of how successful these various efforts were / purport to be would be more helpful.
- throwlaplace 7y ago
  though there's some comparative analysis the point of survey papers is to trace to the history/lineage of current state of the art (it's for researchers after all).
  - thedudeabides5 7y ago
    Yeah I guess, still seems a little odd. Like imagine a 'survey' of health care experiments designed to cure cancer, but no mention anywhere in the paper if any of the experiments worked!
lngnmn1 7y ago
Forecasting based only on past observations is bullshit, like alchemy or tantra.
nl 7y ago
I've discovered a very interesting thing, which apparently most in the machine learning space don't realise (I didn't until recently). Trading isn't what most people think it is at all. I've spent a week or two doing a bit of reading into what it actually is. Three things that have jumped out at me: 1) Trading is not investing. I was extremely doubtful about a lot of the "models" people throw around in this area because to me they seemed closer to numerology than anything related to investment. Then someone pointed out that trading is not investing, and suddenly I looked at these models in a new light. 2) There's lots of interesting trading you can do even without having any idea about what the price is going to do. Do you think volatility is going to increase? You can make money from that - read about the Collar Trade Strategy (This is just an example). 3) There's lots of strategies which don't make enough money for companies to be interested in, but are viable for an individual.
- _ink_ 7y ago
  Three sounds interesting. Could you provide some link/books to read more about that topic?
  - nl 7y ago
    I don't have anything great. I've been reading "The Way of the Turtle", which was pretty much a random pick. The collar trade is explained here: https://www.theoptionsguide.com/the-collar-strategy.aspx https://www.theoptionsguide.com/the-collar-strategy.aspx
- lordgrenville 7y ago
  > Trading is not investing Keynes' famous metaphor is a competition where you have to pick the most beautiful women from a page of photos. The point is not to guess your honest opinion, but to try to predict the aggregate of people's opinions - the face with the most mainstream attractiveness. A speculator needs to guess what other people think, and which way the price will go. By contrast investing is putting money in something that seems to you to have inherent value. https://en.wikipedia.org/wiki/Keynesian_beauty_contest https://en.wikipedia.org/wiki/Keynesian_beauty_contest
roenxi 7y ago
On page 18, 36 and 37 this paper is talking about time horizons of a few seconds to about a month. Over that time horizon the market is going to be behaving much more like a casino than a weighing machine and the people who make money are presumably those who move quickly or know something others don't. I suppose it seems implausible to me that something on arxiv is going to secure an actual advantage over other traders or reliably deduce knowledge that isn't public. Statistics can sometimes seem like magic but it can't do the impossible. Notwithstanding that this topic is interesting, is there any reason to think that these models are valuable in practice?
- phyalow 7y ago
  If they were valuable, why would someone give them away for free!
rocqua 7y ago
This is mostly a list counting how popular certain topics are, with lots of references. I am missing any actual 'review' about which methods had more success, which methods show promise, and which 'subjects' seem more amenable to Deep Learning that others.
- scottlocklin 7y ago
  Several of the papers listed there are known (by me anyway) to be complete bullshit. This seems to be "here are search results on google scholar." I actually don't know anyone using deep learning the hedge fund business, other than for screwing around. It's a terrible tool for that sort of thing. And as someone pointed out below; predicting the future is only a small part of what a trading strategy is (for some trading strategies, forecasting is actually the null set).
- chasedehan 7y ago
  This is an academic literature review article. The point of these is to basically list all of the relevant papers, not discuss the relative merits of each. They are designed as a starting point for academic research.

102 points by Bostonian 7y ago

The most important factor for financial time series forecasting is undoubtedly access to clean data. This is what sets Renaissance apart. There’s no need for particularly sophisticated math - they’ve been doing it for 3 decades.

craze3 7y ago
Renaissance? What is that...?
- apollopower 7y ago
  Renaissance Technologies: https://en.wikipedia.org/wiki/Renaissance_Technologies https://en.wikipedia.org/wiki/Renaissance_Technologies Founded by Jim Simons, the "Father" of quantitative research. The firm is famous for their significant year-over-year returns and notorious for only hiring PhDs from mathematics/physics/computer science. There is an interesting book on how Jim Simons created the company and built his team of academics from the ground up, "The Man Who Solved the Market"
  - craze3 7y ago
    Oh yes I remember reading this wiki before. I just didn't remember what the company was called. I even tried to research it by googling "renaissance time series prediction," and still couldn't find it. I don't understand why I was downvoted? I don't think there was enough context for anyone (who's not in the field) to know what you meant, and I thought it would be useful for people like myself to get clued in.
    - aloukissas 7y ago
      This is the world of HN. If your comments don't fit in the cool crowd, you'll be downvoted.
  - zkid18 7y ago
    Probably Edward Thorp is the "Father" of quantitative research.Since the late 1960s, Thorp has used his knowledge of probability and statistics in the stock market by discovering and exploiting a number of pricing anomalies in the securities markets, and he has made a significant fortune [1] [1] https://en.wikipedia.org/wiki/Edward_O._Thorp https://en.wikipedia.org/wiki/Edward_O._Thorp
deleted 7y ago
[deleted]
ackbar03 7y ago
Are you sure that's the case? I dunno what background your coming from but I did pretty rigorous statistics in college including time series and stochastic processes and all that. Time series are notoriously difficult to forecast. Arch, garch, starch and all that sound good in theory but are pretty crap in practice
apollopower 7y ago
If access to clean data was what set Renaissance apart, we'd most likely see other funds/firms do just as well or Renaissance's returns be much lower. Market data services such as Reuters/Morning Star provide incredibly clean data at great convenience for anything you can imagine, even including astronomical data. Its kind of like the Black-Scholes model. If it really was that simple and straightforward, everybody would be using it to make a ton of money.
- bob_theslob646 7y ago
  >Market data services such as Reuters/Morning Star provide incredibly clean data at great convenience for anything you can imagine, even including astronomical data. Incredibly clean ,that's a very big stretch. Source: have worked with TR sirca db for almost a decade.
throwawaymath 7y ago
That's not what sets Renaissance apart. Please don't add to the baseless speculation that gets tossed around about RenTech so often. Are you aware of just how many trading firms have access to phenomenally clean data? It drives me nuts when I see words like "undoubtedly" thrown around so confidently this way. A new book comes out about Simons and Renaissance, it enters the financial zeitgeist for a little while, and now everyone is apparently an expert on the firm's differentiating competency. For what it's worth, what you're saying is contradicted by Nick Patterson. He did not say that Renaissance had access to clean data no one else did. What he said is that in the early days, they spent almost all their time cleaning the data. In any case, that's table stakes these days. All successful quant firms spend time sourcing exceptional data and ensuring it's as polished as possible.
- throwlaplace 7y ago
  >It drives me nuts when I see words like "undoubtedly" thrown around so confidently this way I see this very often on Reddit and here; someone reads a story and then behaves like they're suddenly a prophet whose job it is to inform the rest of us. On Reddit (because the threads are much bigger) I see this kind of repetition promulgation in the same thread. Here it's typically across threads (as you've noticed). I'll never understand what people get out seeming like an authority figure in a completely anonymous forum.
zkid18 7y ago
Could you elaborate the term of "clean financial data" please?

thedudeabides5 7y ago

Seems like a pretty big list of prior papers, which is a nice reference, but some notion of how successful these various efforts were / purport to be would be more helpful.

throwlaplace 7y ago
though there's some comparative analysis the point of survey papers is to trace to the history/lineage of current state of the art (it's for researchers after all).
- thedudeabides5 7y ago
  Yeah I guess, still seems a little odd. Like imagine a 'survey' of health care experiments designed to cure cancer, but no mention anywhere in the paper if any of the experiments worked!

lngnmn1 7y ago

Forecasting based only on past observations is bullshit, like alchemy or tantra.

nl 7y ago

I've discovered a very interesting thing, which apparently most in the machine learning space don't realise (I didn't until recently). Trading isn't what most people think it is at all. I've spent a week or two doing a bit of reading into what it actually is. Three things that have jumped out at me: 1) Trading is not investing. I was extremely doubtful about a lot of the "models" people throw around in this area because to me they seemed closer to numerology than anything related to investment. Then someone pointed out that trading is not investing, and suddenly I looked at these models in a new light. 2) There's lots of interesting trading you can do even without having any idea about what the price is going to do. Do you think volatility is going to increase? You can make money from that - read about the Collar Trade Strategy (This is just an example). 3) There's lots of strategies which don't make enough money for companies to be interested in, but are viable for an individual.

_ink_ 7y ago
Three sounds interesting. Could you provide some link/books to read more about that topic?
- nl 7y ago
  I don't have anything great. I've been reading "The Way of the Turtle", which was pretty much a random pick. The collar trade is explained here: https://www.theoptionsguide.com/the-collar-strategy.aspx https://www.theoptionsguide.com/the-collar-strategy.aspx
lordgrenville 7y ago
> Trading is not investing Keynes' famous metaphor is a competition where you have to pick the most beautiful women from a page of photos. The point is not to guess your honest opinion, but to try to predict the aggregate of people's opinions - the face with the most mainstream attractiveness. A speculator needs to guess what other people think, and which way the price will go. By contrast investing is putting money in something that seems to you to have inherent value. https://en.wikipedia.org/wiki/Keynesian_beauty_contest https://en.wikipedia.org/wiki/Keynesian_beauty_contest

roenxi 7y ago

On page 18, 36 and 37 this paper is talking about time horizons of a few seconds to about a month. Over that time horizon the market is going to be behaving much more like a casino than a weighing machine and the people who make money are presumably those who move quickly or know something others don't. I suppose it seems implausible to me that something on arxiv is going to secure an actual advantage over other traders or reliably deduce knowledge that isn't public. Statistics can sometimes seem like magic but it can't do the impossible. Notwithstanding that this topic is interesting, is there any reason to think that these models are valuable in practice?

phyalow 7y ago
If they were valuable, why would someone give them away for free!

rocqua 7y ago

This is mostly a list counting how popular certain topics are, with lots of references. I am missing any actual 'review' about which methods had more success, which methods show promise, and which 'subjects' seem more amenable to Deep Learning that others.

scottlocklin 7y ago
Several of the papers listed there are known (by me anyway) to be complete bullshit. This seems to be "here are search results on google scholar." I actually don't know anyone using deep learning the hedge fund business, other than for screwing around. It's a terrible tool for that sort of thing. And as someone pointed out below; predicting the future is only a small part of what a trading strategy is (for some trading strategies, forecasting is actually the null set).
chasedehan 7y ago
This is an academic literature review article. The point of these is to basically list all of the relevant papers, not discuss the relative merits of each. They are designed as a starting point for academic research.