3 ms·

I do not wish to be negative, I just think it is very hard to really judge this system without knowing the technical details behind it. Since we are currently i

by hmate9 10y ago

I do not wish to be negative, I just think it is very hard to really judge this system without knowing the technical details behind it. Since we are currently in a bull market it could just be a fluke. I did not read anything about backtesting. Have you done that? If so, I take it back :)

sebyddd 10y ago
Completely agree with you. The amount of backtesting has been very limited and the time frame was pretty short, so there are definitely changes this is a fluke. Confirming this is my main goal at the moment and I don't take any credit for creating something revolutionary.
- hmate9 10y ago
  Good luck! I am rooting for you.
brianwawok 10y ago
Backtesting does not mean much. It's easy to make an algo do well yesterday. Very hard to make it do well tomorrow.
- Miner49er 10y ago
  True, but it would at least test OP's algo in some bearish market conditions. If no backtesting was done then it has only been tested in the current bull market.
- rtstreak 10y ago
  Indeed, backtesting wouldn't help much in stock prediction. It could work for static input data, but many other factors are involved here.
- hmate9 10y ago
  I completely disagree. Backtesting matters, especially in HFT. What would you rather have: an algorithm that has performed well on 5 years worth of historical data or one that has performed well in the past 2 weeks.
  - sgt101 10y ago
    Neither; these are techniques applied on observed data in an obviously dynamic open domain. The data is uncontrolled - so we have no idea if it is representative of the current state of the domain theory - does it cover the distributions properly; we don't know. We know the domain is dynamic in that the world economy moves like around alot and we can't predict these movements, in the sense that most actors did not call the last substantial negative market movement. The domain is open in that new features can appear that drive the value of the variable of interest; for example the availability of property in China, new battery technology, a patent on nuclear fusion. We do not have techniques that account properly for any of these things. You may as well draw lines on charts and sell that advice, you'd probably get as good a hearing in any Investment Bank in the world ! oh.... wait...
  - brilliantcode 10y ago
    He's not wrong, backtesting really is not the same as testing on real live data. You are fitting your models on data that has already been done, without any of the risks factored in. Even if you do find a system that picks every ups and downs in the backtest, it will do poorly with live trades because the volatility and risk is something that changes based on fundamental data. However, in this case, the fact that no backtest or any sort of risk/reward ratios were published suggests high level of skepticism. tl;dr: past prices can't predict future prices.
  - kasey_junk 10y ago
    In my experience? The later. Regime change is a killer in trading algos. Further, accurate back testing is very difficult for lots of reasons, the most obvious being your own actions aren't reflected. Most algo firms use historical data to develop a hypothesis & then many different ways to validate that hypothesis. Test trading being the one with the most weight.
- tunesmith 10y ago
  Does backtesting work better if you, say, start with seven years of data, use the first five to judge the strategy, and then test it against the last two? Is that a good way to avoid fitting the data?
  - creeble 10y ago
    It very much "depends". I'll give you an example. A friend had a technique that worked very well on the NASDAQ 100 with roughly those BT criteria. One problem was overlooked: The NASDAQ 100 changes over that time. So you aren't necessarily investing in the same securities in year 6 as you were in year 5, even though you're investing in the same "top 100". IOW, backtesting in this particular case had future knowledge that wasn't easy to see. When he started (paper) trading in the real NASDAQ 100, the returns were negative. In that he was lucky: They could have been positive for some time, until the constituent stocks changed.
  - brianwawok 10y ago
    No. When you trade in a live market, your actions cause responses. When backtesting your actions get no responses.

by hmate9 10y ago

sebyddd 10y ago

Completely agree with you. The amount of backtesting has been very limited and the time frame was pretty short, so there are definitely changes this is a fluke. Confirming this is my main goal at the moment and I don't take any credit for creating something revolutionary.

hmate9 10y ago
Good luck! I am rooting for you.

brianwawok 10y ago

Backtesting does not mean much. It's easy to make an algo do well yesterday. Very hard to make it do well tomorrow.

Miner49er 10y ago
True, but it would at least test OP's algo in some bearish market conditions. If no backtesting was done then it has only been tested in the current bull market.
rtstreak 10y ago
Indeed, backtesting wouldn't help much in stock prediction. It could work for static input data, but many other factors are involved here.
hmate9 10y ago
I completely disagree. Backtesting matters, especially in HFT. What would you rather have: an algorithm that has performed well on 5 years worth of historical data or one that has performed well in the past 2 weeks.
- sgt101 10y ago
  Neither; these are techniques applied on observed data in an obviously dynamic open domain. The data is uncontrolled - so we have no idea if it is representative of the current state of the domain theory - does it cover the distributions properly; we don't know. We know the domain is dynamic in that the world economy moves like around alot and we can't predict these movements, in the sense that most actors did not call the last substantial negative market movement. The domain is open in that new features can appear that drive the value of the variable of interest; for example the availability of property in China, new battery technology, a patent on nuclear fusion. We do not have techniques that account properly for any of these things. You may as well draw lines on charts and sell that advice, you'd probably get as good a hearing in any Investment Bank in the world ! oh.... wait...
- brilliantcode 10y ago
  He's not wrong, backtesting really is not the same as testing on real live data. You are fitting your models on data that has already been done, without any of the risks factored in. Even if you do find a system that picks every ups and downs in the backtest, it will do poorly with live trades because the volatility and risk is something that changes based on fundamental data. However, in this case, the fact that no backtest or any sort of risk/reward ratios were published suggests high level of skepticism. tl;dr: past prices can't predict future prices.
- kasey_junk 10y ago
  In my experience? The later. Regime change is a killer in trading algos. Further, accurate back testing is very difficult for lots of reasons, the most obvious being your own actions aren't reflected. Most algo firms use historical data to develop a hypothesis & then many different ways to validate that hypothesis. Test trading being the one with the most weight.
tunesmith 10y ago
Does backtesting work better if you, say, start with seven years of data, use the first five to judge the strategy, and then test it against the last two? Is that a good way to avoid fitting the data?
- creeble 10y ago
  It very much "depends". I'll give you an example. A friend had a technique that worked very well on the NASDAQ 100 with roughly those BT criteria. One problem was overlooked: The NASDAQ 100 changes over that time. So you aren't necessarily investing in the same securities in year 6 as you were in year 5, even though you're investing in the same "top 100". IOW, backtesting in this particular case had future knowledge that wasn't easy to see. When he started (paper) trading in the real NASDAQ 100, the returns were negative. In that he was lucky: They could have been positive for some time, until the constituent stocks changed.
- brianwawok 10y ago
  No. When you trade in a live market, your actions cause responses. When backtesting your actions get no responses.