5 ms·

Duke economist: Most research on why investments do well is all wrong

9 points by prakashk 12y ago

g42gregory 12y ago
Maybe I don't quite understand this article but I am having problems with quite a few statements there. For example: “If I try 20 things,” Harvey says, “then there’s a 95 percent chance that I’ve got a false positive. Just by chance, something is going to correlate.” Yes, this is true and we also knew this before from statistics. But what does this have to do with investment research? Is he saying that just because two things are correlated, they are good investment candidates? I certainly hope not. You need Causation, not Correlation, i.e. you need leading indicators of the right kind. Does Causation suffers from the “multiple testing” problem? I suspect, Yes, you might find something that is a leading indicator more easily. I am not sure that it's a problem for investment research though, because Causations are rare and you could use anything you could find. If the authors think that correlations are good enough for investment research, I kind of doubt their qualifications to write about investment research in a first place. :-) Btw, I agree with their conclusion that most investment research accessible to public is bogus. Another example: "Even if a fund does consistently beat the market, it may just be a coincidence. With so many investment funds out there, some will overperform in a way that seems statistically significant just by chance." This is very true as a conclusion, but is Statistics really the cause? If a fair coin continues to come up "heads", this is certainly possible, but such an event becomes increasingly unlikely. Instead blaming statistical effects, an alternative explanations could be that the fund is using "writing insurance" investment strategy. For example, selling naked put options for years in a row. They would get an nice additional income, thus "beating" the market. Until one day, the underlying stock tanks and the hedge fund will loose a lots of money. All the gains in previous years will be wiped out and, yes it was a coincidence that they "outperformed" the market before. Of course, hedge fund does not care much, since they collected fees from you in previous years. My point is that this may not have much to do with statistics, especially if a fund outperform for multiple years in a row. A lighthearted joke (please don't take it seriously) - change the title to: "Duke economist: Most research on why investments do well is all wrong, including the paper I just published!" :-)
- JoeAltmaier 12y ago
  The outperforming-for-4-years fund is statistically insignificant, because it came from a pool of many funds. They could all have been randomly investing; one will outperform by chance. That's the statistical argument: the fund managers had no real information, and in future you can't predict which funds will continue to profit (the only info you're interested in as an investor).
  - g42gregory 12y ago
    Makes sense. Thanks for clarifying this!
  - nichtich 12y ago
    These argument must be taken with a grain of salt. The truth is you don't know how many independent random investing they actually make. The classic argument is that given more than 1000 people there will be someone with a perfect score in 10 coin toss. But the reality is that some investor only tossed 5 times(big bet, long term investment), while some others tossed 100 times(short term, lots of smaller trading). It's much easier to say to a PE fund which only makes a deal every 2 years that its success is pure luck, than say the same thing to a high frequency trading firm who trades every second.
    - JoeAltmaier 12y ago
      Disagree. If you model stocks as random variable, it doesn't matter which random variable you follow. Trading just increases the overhead.
- mannykannot 12y ago
  For the most part, neither the financial industry nor economics are particularly interested in causal theories. If an investment manager or an economist finds a correlation that looks reasonably robust, they are not going to wait for causality to be established before taking a position (in the former case) or publishing (in the latter case). The difficulty in performing controlled tests makes establishing causality difficult. Furthermore, numerical results look so much more certain than vague hand-waving causal explanations. I am pretty certain that the problem the prof. is writing about is well-known, and controlled for, in the more sophisticated parts of the financial industry. He seems to acknowledge this, but suggests that the industry may be influenced by sloppy academic work. I think he is exaggerating the importance of academic work here: while the industry may look to academe for ideas, it doesn't take them on faith when real money is involved.
hughdbrown 12y ago
Here is the argument in a nutshell: https://xkcd.com/882/ https://xkcd.com/882/ If you have enough hypotheses, some will be significant by chance alone.

9 points by prakashk 12y ago

Maybe I don't quite understand this article but I am having problems with quite a few statements there. For example: “If I try 20 things,” Harvey says, “then there’s a 95 percent chance that I’ve got a false positive. Just by chance, something is going to correlate.” Yes, this is true and we also knew this before from statistics. But what does this have to do with investment research? Is he saying that just because two things are correlated, they are good investment candidates? I certainly hope not. You need Causation, not Correlation, i.e. you need leading indicators of the right kind. Does Causation suffers from the “multiple testing” problem? I suspect, Yes, you might find something that is a leading indicator more easily. I am not sure that it's a problem for investment research though, because Causations are rare and you could use anything you could find. If the authors think that correlations are good enough for investment research, I kind of doubt their qualifications to write about investment research in a first place. :-) Btw, I agree with their conclusion that most investment research accessible to public is bogus. Another example: "Even if a fund does consistently beat the market, it may just be a coincidence. With so many investment funds out there, some will overperform in a way that seems statistically significant just by chance." This is very true as a conclusion, but is Statistics really the cause? If a fair coin continues to come up "heads", this is certainly possible, but such an event becomes increasingly unlikely. Instead blaming statistical effects, an alternative explanations could be that the fund is using "writing insurance" investment strategy. For example, selling naked put options for years in a row. They would get an nice additional income, thus "beating" the market. Until one day, the underlying stock tanks and the hedge fund will loose a lots of money. All the gains in previous years will be wiped out and, yes it was a coincidence that they "outperformed" the market before. Of course, hedge fund does not care much, since they collected fees from you in previous years. My point is that this may not have much to do with statistics, especially if a fund outperform for multiple years in a row. A lighthearted joke (please don't take it seriously) - change the title to: "Duke economist: Most research on why investments do well is all wrong, including the paper I just published!" :-)

JoeAltmaier 12y ago
The outperforming-for-4-years fund is statistically insignificant, because it came from a pool of many funds. They could all have been randomly investing; one will outperform by chance. That's the statistical argument: the fund managers had no real information, and in future you can't predict which funds will continue to profit (the only info you're interested in as an investor).
- g42gregory 12y ago
  Makes sense. Thanks for clarifying this!
- nichtich 12y ago
  These argument must be taken with a grain of salt. The truth is you don't know how many independent random investing they actually make. The classic argument is that given more than 1000 people there will be someone with a perfect score in 10 coin toss. But the reality is that some investor only tossed 5 times(big bet, long term investment), while some others tossed 100 times(short term, lots of smaller trading). It's much easier to say to a PE fund which only makes a deal every 2 years that its success is pure luck, than say the same thing to a high frequency trading firm who trades every second.
  - JoeAltmaier 12y ago
    Disagree. If you model stocks as random variable, it doesn't matter which random variable you follow. Trading just increases the overhead.
mannykannot 12y ago
For the most part, neither the financial industry nor economics are particularly interested in causal theories. If an investment manager or an economist finds a correlation that looks reasonably robust, they are not going to wait for causality to be established before taking a position (in the former case) or publishing (in the latter case). The difficulty in performing controlled tests makes establishing causality difficult. Furthermore, numerical results look so much more certain than vague hand-waving causal explanations. I am pretty certain that the problem the prof. is writing about is well-known, and controlled for, in the more sophisticated parts of the financial industry. He seems to acknowledge this, but suggests that the industry may be influenced by sloppy academic work. I think he is exaggerating the importance of academic work here: while the industry may look to academe for ideas, it doesn't take them on faith when real money is involved.

hughdbrown 12y ago

Here is the argument in a nutshell: https://xkcd.com/882/ https://xkcd.com/882/ If you have enough hypotheses, some will be significant by chance alone.