3 ms·

David - Thanks so much for taking the time to write this. I think your observations on the analysis are valid. However, I slightly disagree with your 0th/macro

David - Thanks so much for taking the time to write this. I think your observations on the analysis are valid. However, I slightly disagree with your 0th/macro point: You said that, "Un-supported financial market data mining is a pet peeve of mine". Had you said "Un-supported financial market advice is a pet peeve of mine" I would be right there with you. But this is not advice, and I went to lengths to point that out in my blog, but I did a bad job of pointing that out in my comment on this page (I guess that's what I get for rushing). As I say in the introduction on my blog, "I want to know whether history suggests that there is anything to be gained from trying to time the market by buying after a fall". I.e. the motive of this is intellectual curiosity, not to create advice. I published it so that I can get the rich feedback that is so valuable in this process (and which is abundant on HN). Now that we have that subtlety out the way, lets look at the more substantial issues. Let me take these one at a time to explain my thoughts: 1. Yes, you're right, there is a lot of correlation. While there are 1578 months, there are far far few economic cycles since 1871, which is another way to think about the problem you are exposing (also in 2). However, this is the data that we have, and I am taking the approach of 'something is better than nothing'. I would say that what this analysis does do is to answer the question "what would the distribution of returns look like for an investment of x year, using timing strategy y, if employed since 1871 to 2008?". Would you say that is fair? If so, the next question is, "can this be generalized in a way that makes it a useful tool for predicting the future?" My answer would be, "not with significant confidence", which I think is your point - is that right? The next question would then be, "so why bother doing it at all?", and my answer would be, "intellectual curiosity" and maybe, "because it's better than nothing at all", though I can see why you would dispute the latter. 2. This essentially is the same as (1), so again I agree with your observation, but maybe not your conclusion. The "n=139" means that in my sample there were 139 months between 1871 and 2008 where the monthly price was 20%-50% lower than max price from the previous 12-months. I'm fairly sure they're accurate. 3. I'm not sure I understand - please explain further. 4. I think we totally agree here on the statistical validity of the data. Where we differ slightly is the presentation. I have not made any conclusions (again my comment on this page was misleading, but at least provocative :). This is a blog that I am using to get other people to help me answer my questions. It is not a research paper. I talk about what I see, not what I conclude. I looked at what my conclusions would be because I know that when most people read this the easiest way for them to grasp the results is to see a viewpoint of what those results would look like if translated to conclusions. I.e. the "would conclude" comments are a tool to spark debate, which seems to have worked:) That is an easy thing for me to say, I know - it may even sound like a cop-out, but if you critique this post as you would a research paper then you miss the point. 5. I could not agree more. In fact, I make this point often to others. I actually did have a hypothesis, but it wasn't crisply defined, nor did I write it down, nor did I mention it in my blog. All of this is bad practice, I know. I didn't go into this on the post because I wanted to make it more accessible (and shorter), by quickly getting to the results. Again, that's a style choice, and I would never dream of doing it in a research paper. 6. (on publication bias). Again, I wholeheartedly agree. Independent verification is a corner stone of good research - but someone has to go first.. If anyone does ever choose to followup on this then I shall be deeply flattered, and eager to see what comes of it. Thank goodness I'm not making any conclusions.. :) 7. I don't feel bad. I'm grateful that someone has taken time to give detailed feedback.

DavidSJ 18y ago
Tom, You drew conclusions from the data. It's not advice, but the conclusions were unsupported, so I consider it unsupported data mining. I'm not questioning your motives. :) 1. Yes, the appropriate question is whether we can generalize from this data. One should prefer uncorrelated data points when sampling data, which is why I say you only have 26 data points. Is it "better than nothing"? Depends on what conclusions you draw. "Tea leaves" are better than nothing if you just think they're pretty. But if you think they have meaning, then they're worse than nothing. In this case, the study has zero predictive value because there is no scientific result. 2. I understand what the N means, but it's misleading since in statistics one usually thinks of N as referring to the number of independent data points. 3. What I mean is: a "real" data point would be an actual market decline of 20-50%. In your model, some of those declines may be represented only once (say, because there was a market recovery the following month), and others are represented up to 11 times (because there was no recovery for the next 11 months). So not only do you have a (much) smaller sample (by a factor of between 60 and 360 depending on which analysis) than your "N" reports, but you also have a biased sample because each buy signal is represented for a varying amount of time depending on how many months it lasted for. This means that, of the handful of market declines which actually occurred in the last century, you might really be representing an even smaller sample. 4. You did draw the following conclusion: "For 10-year investments, investing when the market is down 20% to 50% is probably going to produce better returns than a investment at a random point in time, but it's a close call". Anyways, thanks for your reply. Sorry if my initial comment was a bit too snide. :)

DavidSJ 18y ago

Tom, You drew conclusions from the data. It's not advice, but the conclusions were unsupported, so I consider it unsupported data mining. I'm not questioning your motives. :) 1. Yes, the appropriate question is whether we can generalize from this data. One should prefer uncorrelated data points when sampling data, which is why I say you only have 26 data points. Is it "better than nothing"? Depends on what conclusions you draw. "Tea leaves" are better than nothing if you just think they're pretty. But if you think they have meaning, then they're worse than nothing. In this case, the study has zero predictive value because there is no scientific result. 2. I understand what the N means, but it's misleading since in statistics one usually thinks of N as referring to the number of independent data points. 3. What I mean is: a "real" data point would be an actual market decline of 20-50%. In your model, some of those declines may be represented only once (say, because there was a market recovery the following month), and others are represented up to 11 times (because there was no recovery for the next 11 months). So not only do you have a (much) smaller sample (by a factor of between 60 and 360 depending on which analysis) than your "N" reports, but you also have a biased sample because each buy signal is represented for a varying amount of time depending on how many months it lasted for. This means that, of the handful of market declines which actually occurred in the last century, you might really be representing an even smaller sample. 4. You did draw the following conclusion: "For 10-year investments, investing when the market is down 20% to 50% is probably going to produce better returns than a investment at a random point in time, but it's a close call". Anyways, thanks for your reply. Sorry if my initial comment was a bit too snide. :)