You're right, backtest overfitting is a problem. At Quantopian, we spend a lot of time on education helping people avoid overfitting like this (https://www.quan

You're right, backtest overfitting is a problem. At Quantopian, we spend a lot of time on education helping people avoid overfitting like this (https://www.quantopian.com/lectures#The-Dangers-of-Overfitting https://www.quantopian.com/lectures#The-Dangers-of-Overfitti...). It is easy to make a backtest look good by tweaking things here or there and that's just textbook overfitting. If you follow good statistical principles for model development based on a sound economic basis, the backtest is more of a sanity check to make sure that whatever signal you have found survives the introduction of market impact. If you have consistency between the statistical aspects of in-sample performance (your backtest) and true out-of-sample performance (paper trading on entirely new data), there as an increased probability that you have found a viable signal. Disclaimer, I work at Quantopian.