15 ms·
Using a Keras Long Short-Term Memory Model to Predict Stock Prices
- levesque 8y agoPretty underwhelming. From the very limited experiment demonstrated, the LSTM seems to be playing catch up rather than actually predicting trends.
- epanchin 8y agoYou say that because the prediction line lags the price line? That’s not playing catch up, if the prediction was made at time 0.
- seandougall 8y agoIt doesn't specifically say as far as I can see, but I'm highly doubtful that the prediction was made at time 0 (i.e. with no seed data). That would literally be a lucky guess.
- minimaxir 8y agoI noticed this as well. Many data science thought pieces don't quantify the results, and it's suspicious that they're not included in detail (typically, the R^2 of a simple stock market prediction model is super low, making it impractical putting actual money on the line as it's barely better than guessing randomly: https://twitter.com/minimaxir/status/1021885939361042432 https://twitter.com/minimaxir/status/1021885939361042432 )
- jefftk 8y ago> barely better than guessing randomly In the stock market something that is consistently better than guessing randomly is very valuable! You don't need much of an edge to make a lot of money. (I'm very skeptical about approaches like the OP though)
- citilife 8y agoThe problem with these types of systems are it's difficult to do backtesting on a single continuous data stream... I write about that here: https://blog.projectpiglet.com/2018/01/perils-of-backtesting/ https://blog.projectpiglet.com/2018/01/perils-of-backtesting... I recommend doing something similar to the original post. Neural networks tend to produce valid looking output for a stock price easily, a random walk does too. You have to find correlations, causations, and then review the results carefully. For reference, I wrote my own financial advisor (not directly utilizing deep learning, that functions relatively well): http://projectpiglet.com/ http://projectpiglet.com/ It works relatively well when checking the "causality" which also has it's limits: https://blog.projectpiglet.com/2018/01/causality-in-cryptomarkets/ https://blog.projectpiglet.com/2018/01/causality-in-cryptoma... EDIT: Added some evidence: * 2016 stocks: https://imgur.com/a/j8YWR https://imgur.com/a/j8YWR * Early 2018 crypto: https://blog.projectpiglet.com/2018/01/30-weekly-returns-using-piglet/ https://blog.projectpiglet.com/2018/01/30-weekly-returns-usi... * Early 2018 stocks: https://twitter.com/AustinGWalters/status/976347632439209985 https://twitter.com/AustinGWalters/status/976347632439209985 * All 2018 on Robinhood only: https://imgur.com/a/2CxEFqI https://imgur.com/a/2CxEFqI * Also (I am lettergram on HN), created: https://hnprofile.com https://hnprofile.com and https://redditprofile.com https://redditprofile.com
- kdma 8y agoInteresting but i am a non believer can you provide figures that show gains through years?
- citilife 8y agoAdded in an edit update - I have more, but just tossed some together for you from 2016 - 2018.
- Scarbutt 8y agoThis type of scam has been done many times for years...
- meesterdude 8y ago> I wrote my own financial advisor (not directly utilizing deep learning, that functions relatively well) I do not see enough data to backup the claims of how well it works. And if you can't provide that - what good is it as a tool?
- rustcharm 8y agoWe'll know if it works when he gets to be a trillionaire.
- rademacher 8y agoThese types of experiments pop up from time to time. You have to compare the performance relative to the single lag error. So the baseline is to use the previous time step to predict the current time step. Ultimately, it's not worthwhile to attempt to predict stock timeseries the SNR is far too low. It's more valuable to attempt to predict a trend, e.g., +1 is there will be upward movement in time window x, -1 if downward, 0 is within some epsilon of the current price.
- sergioisidoro 8y agoI preliminarily did the same thing as in this article a while back (with crypto data, since it's easily accessible), just as a way to learn keras. Somehow my network always learned to output a delay of the input no matter how hard I tried to shape it. I've searched through literature briefly and some examples. Some blog posts even claimed they had good time series prediction when I clearly saw they were having the same problem. What I mean to say is, be critical of what you find online, and be critical of your results. This is definitely not an easy problem, and some argue that it's unsolvable all together :)
- jorgemf 8y agoYou are not alone. With time series that is the easiest solution for any model. That is why you need a lot of work creating your datasets and the experiment setup with time series. To avoid the easy solution and obtain something useful.
- jakeinspace 8y agoNobody's gonna do much better than lagging the market price using a model based purely on historical price data, markets aren't nearly that predictable.
- AznHisoka 8y agoI'm skeptical of any system that proves it can predict stock prices vaguely. I would be less skeptical if it was more humble, and specific in its constraints. For example, could it predict the % probability a stock would up X% if it had N consecutive negative days? Or, how many minutes/hours after earnings are announced is the after-hours price usually indicative of what the price will be the next day? You can't predict everything, but you might be able to have a probabilistic prediction of specific scenarios. though even then I'll still be skeptical.. daily stock movements are totally random.
- joshgel 8y agoI wonder if someone could comment on how AI is used at big investment banks? I assume its much more than technical analysis (like this model). I imagine that NLP might be helpful for quickly ingesting a news feed and then making buy/sell decisions based on the news in less time than humans can read and react?
- AznHisoka 8y agoAs you mentioned, one way is to quickly ingest earnings announcements from press wires/company websites. As soon as the earnings report hits the wire, they want to immediately known how big a miss/beat was it, and whether they should immediately dump it or buy more. Other examples include analyzing the words/sentiments in earning calls, to see if certain words indicate a bearish or bullish signal. For example if a CEO mentions the word "headwinds", does that historically lead to a much better quarter next time? Some other companies do analysis with public/private data. For example, you can use a weather dataset to calculate the average temperature for all cities where there are restaurants owned by Cheesecake Factory in a quarter, and see if it will negatively or positively impact sales.
- this_user 8y agoI don't think you need AI for that, though. There are news feeds that are specifically designed to be machine readable. All you need to do is parse it and compare the actual numbers to the expected ones, then calculate the delta for your pricing model. All of this is pretty simple. The real challenge is being faster with this than the competition.
- AznHisoka 8y agoYou're absolutely right. None of this is AI. Wall Street is notorious for making stuff sound more impressive then it really is.
- narrator 8y agoI saw some guys present who did this with currency trading. They had to retrain the model every night and it didn't trade for the first four hours after the market opened.
- usgroup 8y agoSignal/noise on the market is extremely low, and determining whether you’re predicting something or just data snooping your way to a function that looks like it predicts something is a serious competitive advantage in itself (... because all you need then is novel functions to try). Working out a function that passes a split test is inevitable and easy tbh. That function then making money is highly unlikely. This is especially true if your data sources are public.
- hiker512 8y agoWell, I'm only on mobile. But the problem with stock prices is often that you don't have good data, unless you are working for the big four or whatever. And even then, they are apparently still working on integrating external data in real time. Imagine having the whole Twitter stream, maybe categorized by some network in an automatic way. Also prequential evaluation.
- jamesonthecrow 8y agoRegardless of whether or not this would make anyone money, it's a really nice introduction to forecasting time series using LSTMs. Thanks for the post!
- ElBarto 8y agoIsn't the concensus these days that every bit of information is squeezed out so that the stock price always reflects all that is predictable? In addition, major factors are external and trying to predict further price only based on past price is bound to disappoint.
- dointheatl 8y ago11:15, restate my assumptions: 1. Mathematics is the language of nature. 2. Everything around us can be represented and understood through numbers. 3. If you graph these numbers, patterns emerge. Therefore: There are patterns everywhere in nature.
- Cau5tik 8y agoNow, where did I put that drill?
- all2 8y ago> Everything around us can be represented and understood through numbers. One of the biggest problems facing someone trying to assemble a numerical model of some thing or process is whether or not their model is adequate to describe the phenomena they are observing. This is why psychology remains a statistically modeled science, largely relegated to anecdotal research. (Note that my intent is not to speak ill of psychology and its cohort, only illustrate that just because you can slap numbers on a thing doesn't mean you've described it well. Look at the transition of understanding from Newtonian mechanics to relativity and quantum mechanics. Newtonian mechanics was enough until someone saw Mercury was doing weird things it shouldn't. And before Newtonian mechanics we had a broken description of the solar system with more exceptions than rules. We were throwing numbers at a system and failing, in some part, to describe it with consistency.)
- throwawaymath 8y agoWell said. Having to add increasingly many edge cases to your model is a sign your model is insufficient.
- T-A 8y agohttps://xkcd.com/1570/ https://xkcd.com/1570/
- SyneRyder 8y ago12:50 Press Return (For anyone who didn't get the reference, you have an excellent movie & coding soundtrack to catch up on: https://www.youtube.com/watch?v=ShdmErv5jvs https://www.youtube.com/watch?v=ShdmErv5jvs)
- fredliu 8y agoTotal noob question in this space (algotrading with DL algorithms). What DL is good at, is to automate tasks that are easy for humans (e.g. telling cat from dog, understand a sentence from sound wave, translation, etc) but hard for machines without DL. Now, if a task is even hard/borderline impossible for humans to achieve (e.g. predicting stock market with higher accuracy and consistency), why would we believe DL could do a better job than humans?
- fossuser 8y agoDL isn’t just good at automating human tasks (see AlphaZero and AlphaGo). It can be used to recognize patterns and train to solve problems better than humans. Though in this case since the market is people making predictions about a prediction I’m not sure how much it’d help. There’s probably something about humans using a rough estimate of what the stock has been historically to inform where it could go though so it’s value is probably non-zero.
- fredliu 8y agoBy DL, i meant generic DL techniques like CNN, LSTM (used in this article), which relies on large amount of labeled data to train, and predict on similar data, thus my comment about automation. Alpha Go (Zero) is very specialized for the game of go, not sure how much of its specialized algorithm could transfer to other generic use cases.
- rl3 8y ago>Alpha Go (Zero) is very specialized for the game of go, not sure how much of its specialized algorithm could transfer to other generic use cases. AlphaZero is a generalized successor, and it does just that: https://en.m.wikipedia.org/wiki/AlphaZero https://en.m.wikipedia.org/wiki/AlphaZero
- fredliu 8y agoGood to know it generalized, but seems it's only generalized on board games like problems where you have a problem space to search through, is that understanding correct? If so, it probably won't help in the use cases we are talking about here, right?
- DevX101 8y agoI'm not a purist believer in the efficient market hypothesis. BUT, I doubt there's much alpha to be gained simply from looking at price data which is widely and publicly available. Also keep in mind that markets are dynamic feedback loops so even if this model had an edge, the act of publishing this article would work against you to neutralize that edge in the future. There's a good reason the most successful 'quantitative' trading fund, Rennaisance Technologies is so secretive and subjects its employees to a lifetime NDA/non-compete. If there is edge in the market (and I think there are anomalies that can be traded on), I would assume it to come from correlating proprietary datasets with price. Or datasets that have a high barrier to access/analyze. One example that I'd bet still works in some industries would be counting trucks from a supplier to estimate product demand before the companies announces earnings ;)
- SEJeff 8y agoRennaisance is one large quantitative trading firm, there are other bigger and more successful ones however. I happen to work for another very large and successful quant trading firm.
- a9entroy 8y agoWhich company would that be?
- turingcompeteme 8y agoBigger sure, but can you point to one that has had better performance than the medallion fund?
- zugweg 8y agoDo not minmax scale the data. New data outside ranges seen before will be chopped off. Do make relative (high/low), so patterns you find generalize to other domains. Don't fit complex LSTM models on tiny datasets. Somehow this is the shiny thing all starters jump on, while the most simple models are the most proven and robust. Don't just share such a model without giving a disclaimer ("I never tried this out on real data, and would not invest my own money in this solution"). This is real money, real economies, and you could horribly crash these. That is also why you don't write with authority about how to build a bridge (while you never crossed a bridge you've build before). The only reason for the "clearly showing how powerful LSTM models are" is it responding to test set samples it got wrong, and correcting with a lag. How about some evaluation, preferably on another stock than trained on?
- rl3 8y ago>This is real money, real economies, and you could horribly crash these. That is also why you don't write with authority about how to build a bridge ... Setting aside that first bit of hyperbole, I don't see the danger in the bridge example. I mean, if engineers are building bridges based on information they've gleaned from internet articles, there are larger issues at play. As far as markets, retail traders basically exist and have existed within a massive bubble of misinformation since forever. There's a reason order flow is so valuable.
- zugweg 8y agoML is dangerously overhyped (that is an issue at play). This article uses min max scaling, so it will treat a 30$ stock as if it were 20$. You want someone like that managing your pension fund? I refer to engineering code, because it is clear to me we don't want our bridges build by those not skilled enough to make safe bridges. Yet, it is not so clear with ML/AI, while the potential for damage may be even greater. https://en.m.wikipedia.org/wiki/Financial_Modelers%27_Manifesto https://en.m.wikipedia.org/wiki/Financial_Modelers%27_Manife...
- gcb0 8y agowonder how effective those models will become when everyone starts believing in those models to do trades.
- tjpaudio 8y agoSimple linear regression would work just fine for something rudimentary like this. This using neural nets stuff for everything lately is getting ridiculous.
- vilaca 8y agoWhat about trainnig a nn for linear regression?
- gaius 8y agoOblig XKCD https://xkcd.com/1570/ https://xkcd.com/1570/ Seriously tho', articles like this need to come with a big banner at the top saying warning: do not try this at home.
- mishurov 8y agoQuite the opposite. People in finance are much more advanced in terms of mathematical statistics and numerical methods, they use time series analysis, stochastic calculus, etc. Keras is a high level framework for lazy people or for prototyping maybe, this is for trying at home.
- docker_up 8y agoThese predictions and models work until they don't. And when they don't, they blow up in spectacular fashion. Look at Long Term Capital Management. It had big names behind it and the model/strategy made a lot of money, until I believe the Russian bond crisis caused the markets to act irrationally. Then everything they had invested in quickly went down the toilet, and it was big enough to almost cause a financial crisis.
- ifdefdebug 8y agoIs this graph at the end a one-step-ahead prediction? Because if it is, then it looks very much like a 50/50 success/failure ratio. Which is what I would expect, because otherwise you could extract a signal from it.
- inputcoffee 8y agoThis doesn't take away from the exercise, but I notice the model seems to "predict" the movement just after it occurs. It just lags the current price. If there is no alpha in the price signal, then the rational thing to do is to just stay at the current price, which is what the algo seems to have learned to do.
- mino 8y agoSeriously, how on earth is this the news #1 of the frontpage of HN? Isn't this community supposed to know better? Before clicking I thought "I bet they do stacked LSTM to predict the _next_ time step" :|
- java-man 8y agoIt's tough to make predictions, especially about the future. -- Yogi Berra
- samfisher83 8y agoAll these LSTM Stock Market tutorials seem to be a variation of this guys tutorial which he did a while ago: https://github.com/jaungiers/LSTM-Neural-Network-for-Time-Series-Prediction https://github.com/jaungiers/LSTM-Neural-Network-for-Time-Se... Except Jakob explains the folly of the method. Whereas the author of this piece says the following: From the plot we can see that the real stock price went up while our model also predicted that the price of the stock will go up. This clearly shows how powerful LSTMs are for analyzing time series and sequential data.
- spongepoc 8y ago>From the plot we can see that the real stock price went up while our model also predicted that the price of the stock will go up. This clearly shows how powerful LSTMs are for analyzing time series and sequential data. Yes I've noticed on HN recently ML and data science have become popular topics but I'm surprised a post like this has so many votes.
- mmq 8y agoI made this code https://github.com/mouradmourafiq/tensorflow-lstm-regression https://github.com/mouradmourafiq/tensorflow-lstm-regression at least 6 months earlier than the first push in the github repo you mentioned, and I am sure many people did some version before that. A lot of people sometimes just need a reason to play with some technology. I assume that OP used stock market to learn about LSTMs.
- samfisher83 8y agoThe guy I posted uses Keras which is what this guys code looks like. I am sure plenty of people have done lstm on the stock market but the form of the code looked similar between the two.
- halayli 8y agoYou can use whatever prediction technique you like, but if your model is wrong, then so will the prediction. Predicting stock prices requires considering as many factors as you can gather that goes into setting the stock price, and how the factors correlate with each other. > previous price of a stock is crucial in predicting its future price This is a poor and incorrect model. The stock price has plenty other variables and many of which are unknown. What's used here is just a single variable.
- nabla9 8y agoThe author uses stock market as a toy example. He is not actually attempting to do serious forecasting. Only comparing the prediction accuracy against simpler models like ARIMA or VARMAX (as the author suggest) can tell you if the model has any use.
- currymj 8y agoIt seems like if you do anything with financial data like this, people often aren’t capable of accepting that it’s just a toy example — they judge it as if it must be intended to really make money. I don’t know why.
- eternalny1 8y agoI used to run a technical analysis site for exchange traded funds (ETFs). Skipping to the "results" of this experiment, I am seeing nothing. It looks like a slow, short period moving average (SMA)... which does nothing in terms of predicting anything. It just averages out the chaos over time.
- myth_buster 8y agoThink the important thing is model performance and tuning hyper parameters which is missing from this tutorial. Without that you are just regurgitating some model that does something.
- goodside 8y agoThis is pure nonsense. This isn’t even the right way to begin thinking about this as a forecasting task — the target series should be log-normal returns, not raw asset price. The performance of this model is laughably bad, which is probably why he spends zero time evaluating its effectiveness. You could trivially get better forecasts than this by naively repeating the last-observed price. This isn’t ML. It’s cargo-cult performance of words and ideas that ML people use.
- zz34 8y agoWelcome to ML in 2018.
- jwiley 8y agoCan you clarify why "the target series should be log-normal returns" is important or provide a pointer for more information?
- jpeterson 8y agoInvestment markets operate on relative gain, not absolute gain. E.g.: if you invest in a stock and it gains $5, this would be a great return for a $1 stock but a poor one for a $1000 stock, so the absolute gain doesn't mean anything on its own. A 5% return always means that you've gained 5% on your investment.
- ak39 8y agoWould it be the same (valid) with percentage returns?
- deleted 8y ago[deleted]
- goodside 8y agoLess so. Log-normal returns are better because they have the property that a summation of log-normal returns over contiguous intervals is equal to the log-normal returns of the combined interval. In other words: Losing 5% and then gaining 5% doesn’t put you back at exactly 100%, and log-normal fixes that.
- zz34 8y agoThis is complete rubbish. Zero evaluation compared to any sane baseline, zero thinking about representation (predicting raw price?!). I'll just say it, ML was frankly better when fewer people knew it and there was less money in claiming to be an expert.
- cam3ham 8y agolol Derrick Mwiti getting rekt in the comments
- ttlei 8y agoFrom my experience, LSTM or other recurrent neural network models only work "well" at forecasting bounded and periodic or oscillating time series. Might work for something like seasonal sale data, but would fail spectacularly with unbounded and chaotic time series like stock prices.
- gcmartinelli 8y agoI believe a simple moving average would achieve the same result and that mixing training and test data for validation invalidates your model results... On a positive note, nice showcase of how simple implementation with Keras is.
- hmate9 8y agoI don’t know what’s more embarrassing. This article, or the fact that it’s #1 on HN right now.
- btbuildem 8y agoAgain, as always, and every time -- if a stock market price-predicting scheme worked, it would not be published on the internet for free. It would not be for sale. It would be in constant use to make money for the authors.
- ericb 8y agoPredicting stock prices with deep learning is the "Killing Baby Hitler" of Time Travel. Even though they know it won't work out, everyone tries it their first time.
- mbrumlow 8y agoShame on me for looking. But this is a account that post only from fritz.ai -- might be nonsense spam.
- i_phish_cats 8y agoLet's take it easy on the author... LSTM's are cool especially when you first grok them, and stock data is the most abundant time series data out there.
- ipsa 8y agoIt's the webdev equivalent of creating a log-in form tutorial and putting the password in a JavaScript variable.
- aj7 8y agoFeed it 3 cycles of a perfect sine wave. What does it predict?
- drngdds 8y agoI know nothing about ML and not very much about finance, but I find it incredibly unlikely that some random guy on Medium has an algorithm that can be used to accurately predict the market. Why isn't he hilariously rich?
- truculent 8y agoMy favourite part of this is that Francois Chollet (the author of Keras) specifically warns against this exact use of LSTMs in his book (Deep Learning with Python)
- williesleg 8y agoYou can't predict the future of a stock price based on the past price. Future price is based on future potential. If you could predict it based on the past, everyone would be rich. Instead, the only ones getting rich are those selling the data.
- chunsj 8y agoNo, this is a bad example of prediction; if you bet money on the direction of prediction, you'll lose much. Drawing prices and making it looking as similar is not correct prediction. At least you should show >50% correct direction prediction if it is considered to be "well processed" machine learning.
- throwaway487548 8y agoWhy on earth there is supposed to be even a correlation between so-called past and future (bullshitting as a business aside)? It is like predicting Trump by Nickson or, even better - predicting a next Trump.
- jaupe 8y agoI cannot fault your enthusiam but I can fault your method :) Read Tsay's Financial Time Series Analysis for a better idea how to forecast financial data. The biggest problem you will face is stationarity: the statistical properties of the data is not constant over time. For example, the mean and std dev is not constant over time. Using returns instead of raw prices helps to make better financial forecasts. Two methods to explore: 1. You are better off predicting stock prices by predicting future returns and then forecasting is the current price plus predicted future return. 2. You could use your neural model to predict absolute size of returns using realized volatility.
- norswap 8y ago> From the plot we can see that the real stock price went up while our model also predicted that the price of the stock will go up. This clearly shows how powerful LSTMs are for analyzing time series and sequential data. Is laughter or tears the right reaction here?