3 ms·
A cautionary note on backtesting (ie, assessing how trading strategies would have performed over a historical period in time). If an a posteriori probability d
by adg001 14y ago
A cautionary note on backtesting (ie, assessing how trading strategies would have performed over a historical period in time).
If an a posteriori probability distribution is a good fit for historical events, it doesn't mean in any way it is going to fit future data points. It may or may not.
Hence, use backtesting with care while trading.
- afarquhar 14y agoyep you're totally right, but hopefully it's still useful!
- startuprob 14y agoAbsolutely true, backtesting is not a guarantee of anything. That being said, I can't remember who said that "No model is perfect but some models are useful."
- phreeza 14y ago"All models are wrong, but some are useful." George EP Box. I put that quote on the front page of my masters thesis.
- frinxor 14y agothis is great. love the design, and I can totally see a broker acquiring this so their clients will make more trades on their backtested-to-be-profitable strategies!
- theorique 14y agoPast performance may not be a guarantee of future performance, but it's all we've got to work with.
- phreeza 14y agoThere are more, statistically sounder methods that could be used to search the space of possible strategies, a la bootstrapping. These could be used to make more meaningful statements about the strategy, if the assumption is that previous statistics predict future statistics, which of course might also be false (black swans, etc.) Given the vast amount of parameters that constitute a strategy, naive optimizing of the outcome of the strategy in the past is bound to produce overfitting in the majority of cases...
- adg001 14y agoActually, this is not accurate. Before relying on our models, we get also the opportunity to validate them by checking at what extent they successfully predicts future data points. Simply for the next t-time periods let the model give the user an estimate of the future prices/rates, together with an estimate of how accurate the model expects these predictions to be. This would allow the user to build confidence in the algorithm strategy she/he came up with, before employing it on the open market.
- theorique 14y agoI mean it in the naive sense - we can observe the past but not the future. It definitely makes sense to paper trade a strategy and see its predictiveness in real time.