6 ms·
This is pure nonsense. This isn’t even the right way to begin thinking about this as a forecasting task — the target series should be log-normal returns, not ra
by goodside 8y ago
This is pure nonsense. This isn’t even the right way to begin thinking about this as a forecasting task — the target series should be log-normal returns, not raw asset price. The performance of this model is laughably bad, which is probably why he spends zero time evaluating its effectiveness. You could trivially get better forecasts than this by naively repeating the last-observed price.
This isn’t ML. It’s cargo-cult performance of words and ideas that ML people use.
- zz34 8y agoWelcome to ML in 2018.
- jwiley 8y agoCan you clarify why "the target series should be log-normal returns" is important or provide a pointer for more information?
- jpeterson 8y agoInvestment markets operate on relative gain, not absolute gain. E.g.: if you invest in a stock and it gains $5, this would be a great return for a $1 stock but a poor one for a $1000 stock, so the absolute gain doesn't mean anything on its own. A 5% return always means that you've gained 5% on your investment.
- ak39 8y agoWould it be the same (valid) with percentage returns?
- deleted 8y ago[deleted]
- goodside 8y agoLess so. Log-normal returns are better because they have the property that a summation of log-normal returns over contiguous intervals is equal to the log-normal returns of the combined interval. In other words: Losing 5% and then gaining 5% doesn’t put you back at exactly 100%, and log-normal fixes that.
- theothermkn 8y agoIn the extreme, two successive trades, where the first gains 110% and the second loses 100%, “average” out to a 5% return. However, you don’t want to make that pair of trades.
- ak39 8y agoAh, that makes sense. Many thanks.
- colinchartier 8y agoI think their point is that predicting that "the stock will go up by 1$" or "the stock will go down by 1$" is worse than "the stock will go up by 0.05%" and "the stock will go down by 0.05%" because of this little paradox: 50$ increase from 50$ is 100% increase 50$ decrease from 100$ is 50% decrease e.g., if the model finds 50$ increase/decreases, that actually corresponds to very different wealth changes
- jdmichal 8y agoWell, just at a minimum... There's the fact that the majority of large gains and losses exist over single-day frames. That is, guessing "right" or "wrong" on movement means little when one slip on the wrong day will decimate your returns.
- sseveran 8y agoHere you go: https://financetrain.com/why-lognormal-distribution-is-used-to-describe-stock-prices/ https://financetrain.com/why-lognormal-distribution-is-used-...
- bigmit37 8y agoThank you.
- hendzen 8y agohttps://en.wikipedia.org/wiki/Stationary_process https://en.wikipedia.org/wiki/Stationary_process
- deleted 8y ago[deleted]
- vl 8y agoEven more so since test data can't be from the same time range: i.e. for time series you need to split train/test by date, not randomly, otherwise your model just memorizes the series.
- ipsa 8y agoThe article splits in time, not randomly.
- goodside 8y agoIt's standard practice to validate forecasts on non-randomized test/validation splits of the same time series, since this simulates the conditions where the model will be deployed in reality: It will know everything there is to know about the past, and it will know nothing about the future. See Hyndman's fpp2 — https://otexts.org/fpp2/accuracy.html https://otexts.org/fpp2/accuracy.html Also, his description of rolling window validation: https://robjhyndman.com/hyndsight/rolling-forecasts/ https://robjhyndman.com/hyndsight/rolling-forecasts/
- Loughla 8y agoAs someone outside the tech sphere on either coast, that's all ML seems to be. What I've seen from companies marketing to Higher Education is - we have a lot of data, you set arbitrary flags to the data that you believe indicate 'x' (or even better, they have pre-built data expectations) and you will get 'y' outcome. And none of it is actually based on anything real. It's all anecdotal applied to extreme amounts of actual data. And when I read about ML on here, it seems to confirm my experiences.
- lvs 8y agoIt's so much worse to see ML applied to problems that could surely be studied and understood mechanistically, but people are sold on using ML instead in deference to buzzwords alone. The result is that they may get some model of a phenomenon, but they'll never learn a goddamn thing about why it works. What good is that?
- ChristianGeek 8y agoIt depends on what your goal is; sometimes it’s the destination, sometimes it’s the journey.
- goodside 8y agoThe problem isn’t that ML has gotten worse. It’s just as rigorous and far more powerful than it ever was. The problem is ML is hard, it hasn’t gotten orders-of-magnitude easier to understand, and there’s enormous incentive now to pass off amateur understanding as complete. The real ML still happens — it’s just drowned out.
- candiodari 8y agoI think people are saying that that's something they're not happy with. The big methods in AI, like backprop, 1) work a LOT better for specific problems than statistics or statistical learning ever has (and at this point, I think we can safely say: ever will) 2) a lot of methods either can't be explained, or outright shouldn't work, according to statistical theory. The use of statistics in machine learning is limited to evaluating performance and individual element performance (and even that is tenuous at best in many cases). If you ask, say, why would an autoencoder, with an LSTM on it's compressed representation and Q-learning evaluation have somewhat decent performance on half the computer games humans ever designed ? Statistics will not be useful in formulating an answer. If you ask extremely valid questions, like "why would an LSTM predict anything ?". Statistics draws a blank. There is no good reason to assume an LSTM will ever converge (and on a truly random dataset, it won't, whereas statistical methods will still allow you to say something). I think there's 2 reasons for this 1) the "upper limit" of complexity a human can understand in a statistical model is lower than the upper limit a neural network can "understand". In statistics the human understanding is critical to getting to a valid model, in machine learning ... it is not. Meaning machine learning can learn relationships a human mind cannot. 2) There must be some fundamental property of the world we live in that matches neural network architecture. In order for backprop to work on real-world problems, it has to be the case that almost all real world phenomena are continuous, both "raw" and in the frequency domain. If this wasn't the case, machine learning would never be able to learn anything.
- Nalta 8y agoClearly one LSTM didn't work. Lets try FOUR! - edit: miscounted number of LSTMs
- snissn 8y agohttp://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MinMaxScaler.html http://scikit-learn.org/stable/modules/generated/sklearn.pre... Should normalize the data and raw prices won’t be used and instead the normalized coefficients are effectively a percentage
- nigealj 8y agoIt's the AI winter we're all fearing!