5 ms·

Algorithmic Trading Using Logistic Regression

38 points by cshad 6y ago

deleted 6y ago
[deleted]
lalaland1125 6y ago
This article is horribly incorrect. All of the numbers and analysis are wrong and nobody should be using it as a guide to anything. The key problem with their analysis is that they don't do a proper train-test split. All of their results are probably just overfitting on the training. You know that they are doing an incorrect train/test split because they are using train_test_split from the sklearn library, which is only designed to work on independent data. However, stock prices are highly correlated with each other. In order to get a true "independent" test set you have to do a time split where all the test set values are from the future and all of the training set values are from the past.
- bko 6y ago
  My bigger problem with the article is he’s fishing. He’s just throwing a bunch of technical parameters against all stocks and testing a few randomly selected time periods. Of course he’ll find something especially when he can wave off a model by saying oh it doesn’t work on X Reminds me of https://xkcd.com/882/ https://xkcd.com/882/
  - fractionalhare 6y ago
    That's right. At minimum there should be Bonferroni correction to adjust for the number of (implicit) hypothesis tests being executed. If we pursue this methodology with a significance level of p = 0.05 and test 20 independent signals against some financial metric, we can expect to find one of them to exhibit correlation simply by chance. Considering the OP tested 32 technical indicators against the dependent variable, this likely happened.
- wrinklytidbits 6y ago
  Agreed. For those interested in time series analysis I just finished a small project with the help of Introduction to Time Series Forecasting with Python by Jason Brownlee. The relevant section is 24.4 Rolling Forecast ARIMA Model.
- fractionalhare 6y ago
  > The key problem with their analysis is that they don't do a proper train-test split. Yes, that's bad. But the enterprise is doomed even earlier than that: the author is using a naive regression technique on timeseries data. Stock price data is typically log-normally distributed and non-stationary. If the author is not going to use a robust timeseries technique like ARIMA, they need to transform and constrain the data to account for non-stationarity. Otherwise it's trivial to find superfluous correlations which appear to show a strong linear relationship between some fundamental financial metric and some other technical indicator, because both may have a temporal component (e.g. stocks tend to go up over time, and a lot of technical indicators have some implicit component which also increase or decrease with time). This is also why it's important to have a strong fundamental thesis to explain what's going on.
  - ulucs 6y ago
    I don't think throwing ARIMA without the proper knowledge would be enough. AR is evident, but I and MA will be unjustified assumptions that take away from the meat of the subject. Also you might need to get into Panel Data methods when you are evaluating more than one stock
- darawk 6y ago
  This is a good general point, but this sort of thing is probably not an issue for the types of indicators he's using, and the simple regression models he's using. The model is not sufficiently complex to learn regime patterns, which is the problem you're alluding to. This would be an issue if he was using a neural net, though. I agree with your broader point that this analysis is bad, though. Another criticism is that he's predicting class labels, but not accounting for magnitude. It is possible to be able to predict the direction of the market tomorrow, with say, 60% accuracy, and still not be able to make money. If you can predict up/down with 60% accuracy, and on average when you're right you make 1%, and when you're wrong you lose 10%, you still have a bad strategy.
- emteycz 6y ago
  Looking at the replies here, I wonder if this was application of that meme where to get the right answer, you need to say the wrong one convincingly to get others to tell you how wrong you are.
- smabie 6y ago
  The site is down for whatever reason, but daily return data isn't really highly correlated. The auto-correlation and partial auto-correlations are close to zero regardless of the lag you use. Monthly data shows slight acf and pacf for the previous period for most equities. Moreover, if you are just looking at the direction of daily movements (up or down), the series of moves fits a geometric distribution very well, with a slight bias in the up probability than the down one. But regardless, the fix you're suggesting is correct. You can't just take random days out of order. Even though daily returns show low acf/pacf, stocks show trends in the magnitude of their returns, but not necessarily their direction. For example, a given stock could be popping off with strong upward momentum, and still be down half the days. It's just that the returns are greater on the upside than downside. You see this kind of incorrect thinking a lot with people who don't work in quant finance: they try and predict the direction and sometimes magnitude of returns. In real life, no one actually does this, as it is practically impossible. Instead, you try and construct a portfolio in which you are exposed to some factor in which you think is correlated to out-performance. I personally think believe that forecasting direction and magnitude to be literally impossible with ML methods in daily data. I (and everyone else) has come to this conclusion after many failed attempts with every ML/AI tool in existence. Fortunately, it turns out you don't really need to forecast returns to deliver alpha and superior risk-adjusted returns. It's just the most obvious thing to do for people new to finance, but experienced with ML.
  - cshad 6y ago
    Hi, I'll have to look into why the site crashed. Thank you for the feedback. I still have a lot to learn and your reply definitely helps put things in perspective for me. Will look to use this information moving forward.
- cshad 6y ago
  Hey thanks for pointing this out! Still new to this, definitely a huge mistake. Will correct moving forward!
Res563 6y ago
He also has an article up called How I Lost All of My Money in the Stock Market He is going to lose it all again if the rest of his algos look like this one. https://handsoffinvesting.com/how-i-lost-all-of-my-money-in-the-stock-market/ https://handsoffinvesting.com/how-i-lost-all-of-my-money-in-...
- deleted 6y ago
  [deleted]
smabie 6y ago
Okay, so I got the page to load (in a previous comment, I said it would not load). This guy is doing nothing correctly, and his code is an absolute mess. He's not using pandas except for displaying data and reading a CSV and his code is unnecessarily ugly and verbose. You really shouldn't be using very many loops at all when doing quant research. And he capitalizes all variable names, but whatever I guess. He's using raw price data (not sure if Yahoo split and dividend adjusts, so that might be another problem), not log returns. Also as lalaland1125 pointed out, he's literally splitting his data into training and testing randomly, not keeping contiguous time-series. This makes no sense, and no quant researcher would ever do this. Because of these problems, his results are completely and totally invalid and readers should completely ignore this article. Looking at his linkedin, he has no experience in finance, as I expected. You can't just take over your textbook ML models and apply them to finance. It doesn't work like that. I've seen a lot of these ML stock market witchcraft kind of medium posts lately and I hope they're making money off of medium, because they definitely aren't developing successful strategies. Edit: I wasn't going to mention this, but I run a no bullshit quantitative finance blog: https://cryptm.org/ https://cryptm.org/ I keep it mathematically rigorous and honest. Some of it is original research, but most of it is backtesting/analysis of well known academic research. Edit2: Take a look at this code: while x < (len(data)): if x < (len(data)-5): if ((close_prices[x+1] + close_prices[x+2] + close_prices[x+3] + close_prices[x+4] + close_prices[x+5])/5) > close_prices[x]: Five_Day_Obs.append(1) else: Five_Day_Obs.append(0) else: Five_Day_Obs.append(0) x+=1
- cshad 6y ago
  Hey, thanks for pointing out some of the mistakes in my code. I'm definitely new to this field and still have a lot to learn. It's great to get feedback like this! I would love to connect with you and hear more about the work you are doing if you have time. I reached out on LinkedIn!

38 points by cshad 6y ago

deleted 6y ago

[deleted]

This article is horribly incorrect. All of the numbers and analysis are wrong and nobody should be using it as a guide to anything. The key problem with their analysis is that they don't do a proper train-test split. All of their results are probably just overfitting on the training. You know that they are doing an incorrect train/test split because they are using train_test_split from the sklearn library, which is only designed to work on independent data. However, stock prices are highly correlated with each other. In order to get a true "independent" test set you have to do a time split where all the test set values are from the future and all of the training set values are from the past.

bko 6y ago
My bigger problem with the article is he’s fishing. He’s just throwing a bunch of technical parameters against all stocks and testing a few randomly selected time periods. Of course he’ll find something especially when he can wave off a model by saying oh it doesn’t work on X Reminds me of https://xkcd.com/882/ https://xkcd.com/882/
- fractionalhare 6y ago
  That's right. At minimum there should be Bonferroni correction to adjust for the number of (implicit) hypothesis tests being executed. If we pursue this methodology with a significance level of p = 0.05 and test 20 independent signals against some financial metric, we can expect to find one of them to exhibit correlation simply by chance. Considering the OP tested 32 technical indicators against the dependent variable, this likely happened.
wrinklytidbits 6y ago
Agreed. For those interested in time series analysis I just finished a small project with the help of Introduction to Time Series Forecasting with Python by Jason Brownlee. The relevant section is 24.4 Rolling Forecast ARIMA Model.
fractionalhare 6y ago
> The key problem with their analysis is that they don't do a proper train-test split. Yes, that's bad. But the enterprise is doomed even earlier than that: the author is using a naive regression technique on timeseries data. Stock price data is typically log-normally distributed and non-stationary. If the author is not going to use a robust timeseries technique like ARIMA, they need to transform and constrain the data to account for non-stationarity. Otherwise it's trivial to find superfluous correlations which appear to show a strong linear relationship between some fundamental financial metric and some other technical indicator, because both may have a temporal component (e.g. stocks tend to go up over time, and a lot of technical indicators have some implicit component which also increase or decrease with time). This is also why it's important to have a strong fundamental thesis to explain what's going on.
- ulucs 6y ago
  I don't think throwing ARIMA without the proper knowledge would be enough. AR is evident, but I and MA will be unjustified assumptions that take away from the meat of the subject. Also you might need to get into Panel Data methods when you are evaluating more than one stock
darawk 6y ago
This is a good general point, but this sort of thing is probably not an issue for the types of indicators he's using, and the simple regression models he's using. The model is not sufficiently complex to learn regime patterns, which is the problem you're alluding to. This would be an issue if he was using a neural net, though. I agree with your broader point that this analysis is bad, though. Another criticism is that he's predicting class labels, but not accounting for magnitude. It is possible to be able to predict the direction of the market tomorrow, with say, 60% accuracy, and still not be able to make money. If you can predict up/down with 60% accuracy, and on average when you're right you make 1%, and when you're wrong you lose 10%, you still have a bad strategy.
emteycz 6y ago
Looking at the replies here, I wonder if this was application of that meme where to get the right answer, you need to say the wrong one convincingly to get others to tell you how wrong you are.
smabie 6y ago
The site is down for whatever reason, but daily return data isn't really highly correlated. The auto-correlation and partial auto-correlations are close to zero regardless of the lag you use. Monthly data shows slight acf and pacf for the previous period for most equities. Moreover, if you are just looking at the direction of daily movements (up or down), the series of moves fits a geometric distribution very well, with a slight bias in the up probability than the down one. But regardless, the fix you're suggesting is correct. You can't just take random days out of order. Even though daily returns show low acf/pacf, stocks show trends in the magnitude of their returns, but not necessarily their direction. For example, a given stock could be popping off with strong upward momentum, and still be down half the days. It's just that the returns are greater on the upside than downside. You see this kind of incorrect thinking a lot with people who don't work in quant finance: they try and predict the direction and sometimes magnitude of returns. In real life, no one actually does this, as it is practically impossible. Instead, you try and construct a portfolio in which you are exposed to some factor in which you think is correlated to out-performance. I personally think believe that forecasting direction and magnitude to be literally impossible with ML methods in daily data. I (and everyone else) has come to this conclusion after many failed attempts with every ML/AI tool in existence. Fortunately, it turns out you don't really need to forecast returns to deliver alpha and superior risk-adjusted returns. It's just the most obvious thing to do for people new to finance, but experienced with ML.
- cshad 6y ago
  Hi, I'll have to look into why the site crashed. Thank you for the feedback. I still have a lot to learn and your reply definitely helps put things in perspective for me. Will look to use this information moving forward.
cshad 6y ago
Hey thanks for pointing this out! Still new to this, definitely a huge mistake. Will correct moving forward!

Res563 6y ago

He also has an article up called How I Lost All of My Money in the Stock Market He is going to lose it all again if the rest of his algos look like this one. https://handsoffinvesting.com/how-i-lost-all-of-my-money-in-the-stock-market/ https://handsoffinvesting.com/how-i-lost-all-of-my-money-in-...

deleted 6y ago
[deleted]

smabie 6y ago

Okay, so I got the page to load (in a previous comment, I said it would not load). This guy is doing nothing correctly, and his code is an absolute mess. He's not using pandas except for displaying data and reading a CSV and his code is unnecessarily ugly and verbose. You really shouldn't be using very many loops at all when doing quant research. And he capitalizes all variable names, but whatever I guess. He's using raw price data (not sure if Yahoo split and dividend adjusts, so that might be another problem), not log returns. Also as lalaland1125 pointed out, he's literally splitting his data into training and testing randomly, not keeping contiguous time-series. This makes no sense, and no quant researcher would ever do this. Because of these problems, his results are completely and totally invalid and readers should completely ignore this article. Looking at his linkedin, he has no experience in finance, as I expected. You can't just take over your textbook ML models and apply them to finance. It doesn't work like that. I've seen a lot of these ML stock market witchcraft kind of medium posts lately and I hope they're making money off of medium, because they definitely aren't developing successful strategies. Edit: I wasn't going to mention this, but I run a no bullshit quantitative finance blog: https://cryptm.org/ https://cryptm.org/ I keep it mathematically rigorous and honest. Some of it is original research, but most of it is backtesting/analysis of well known academic research. Edit2: Take a look at this code: while x < (len(data)): if x < (len(data)-5): if ((close_prices[x+1] + close_prices[x+2] + close_prices[x+3] + close_prices[x+4] + close_prices[x+5])/5) > close_prices[x]: Five_Day_Obs.append(1) else: Five_Day_Obs.append(0) else: Five_Day_Obs.append(0) x+=1

cshad 6y ago
Hey, thanks for pointing out some of the mistakes in my code. I'm definitely new to this field and still have a lot to learn. It's great to get feedback like this! I would love to connect with you and hear more about the work you are doing if you have time. I reached out on LinkedIn!