4 ms·

Machine Learning Often a Complicated Way of Replicating Simple Forecasting

101 points by henning 8y ago

fleshfly 8y ago
Is the author suggesting that we should think twice about running complicated machine learning models on data without understanding the data first, and for which financial econometricians already have a vast set of tools to analyze it in an intuitive and parsimonious way? Rubbish!
stanfordkid 8y ago
Pretty banal article. TLDR: A very complicated model will fit simple data, as will a simple model. Modern ML/AI has nothing to do with 1D timeseries forecasting.
- cs702 8y ago
  Was about to write the same thing. What the heck is this doing on the front page?
  - smus 8y ago
    Hacker news jumps at any opportunity to write off ML
    - dang 8y ago
      Please don't post unsubstantive comments here.
  - henning 8y ago
    There have been thousands of papers and many books written on the subject of using machine learning for time series forecasting and an unknown amount of money traded/invested with it. Most of them cast trading as a time series forecasting problem where performance is measured by mean square error. They usually then assume that would lead to profitable trading. You don't get to define what "modern machine learning" is any more than I do. The application of complex models to higher-dimensional domains have problems of their own. Both the successes and failures are worth writing and thinking about. The post is an important reminder about simplicity and complexity in modeling. That's why I submitted it.
    - stanfordkid 8y ago
      I don't understand why I'm being downvoted for this opinion -- the topic of overfitting gets beaten to death by the 3rd week of every intro AI/statistics/ML class out there. I didn't find it very worthy of the frontpage (or a blog post).
PLenz 8y ago
Yes, in a toy space ML replicates simplistic measures, there's only so much information to be had in such a space to start with. The power of these techniques is when we apply them to high-complexity or even unbounded-complexity areas.
- candiodari 8y ago
  And that you apply them to domains where the knowledge of the domain is basic -at best. It enables people who know one subject well -machine learning- to do speech recognition and weather forecasting well. Plus this article has a number of problems. It compares machine learning to MA(3). That it works pretty well totally ignores the selection of MA(3) vs MA(5). MA(100) is "common" indicator often talked up (why I do not know, except perhaps it protects you from really big errors, but other than that it's not very helpful). If you actually try to use the prediction of MA(3) you will find it is often very hard to use, because it's always just too late. I wonder if the machine learning version had the same problem, or not.
  - afro88 8y ago
    > If you actually try to use the prediction of MA(3) you will find it is often very hard to use, because it's always just too late. I wonder if the machine learning version had the same problem, or not. Judging from the graph in the article, it looks like it doesn't. The model didn't reinvent the moving average, it invented it without the inherent delay. Though it does seem to diverge from real price over time.
  - longerthoughts 8y ago
    >If you actually try to use the prediction of MA(3) you will find it is often very hard to use, because it's always just too late. I wonder if the machine learning version had the same problem, or not. Why does the sensitivity matter when we know that the accuracy was so similar? The point on sensitivity does make me wonder how an exponential moving average would've performed though.
    - candiodari 8y ago
TTPrograms 8y ago
"Is advanced machine learning a very complicated way of reproducing the results of trivial forecasting methods? In many cases it appears this is the case." If by "many cases" you mean "the one example I looked at which was carried out by an undergrad" than sure. That sort of phrasing really gives away the author's bias.
- longerthoughts 8y ago
  The author isn't arguing that ML isn't useful, they're simply pointing out that it isn't worthwhile for all problems and data sets.
m3kw9 8y ago
But the power comes when the model doesn’t overfit and can predict way more than your simple models
longerthoughts 8y ago
Why are so many people reading this as an attack on ML? The author is simply reminding people that ML isn't a universal remedy for uncertainty. "Stop writing ML models solely for the sake of writing ML models" is a very different message than "ML didn't improve upon MA(3) in this case so it's universally worthless".
iotasigma 8y ago
I understand that the general sentiment for this article seems to be dismissive but I want to share this awesome University of Vermont lecture series/course. It's related to forecasting mathematics and includes an introduction to fractal geometry which I haven't seen much of elsewhere. https://youtu.be/Lr3qrB6dPCM https://youtu.be/Lr3qrB6dPCM

101 points by henning 8y ago

Is the author suggesting that we should think twice about running complicated machine learning models on data without understanding the data first, and for which financial econometricians already have a vast set of tools to analyze it in an intuitive and parsimonious way? Rubbish!

stanfordkid 8y ago

Pretty banal article. TLDR: A very complicated model will fit simple data, as will a simple model. Modern ML/AI has nothing to do with 1D timeseries forecasting.

cs702 8y ago
Was about to write the same thing. What the heck is this doing on the front page?
- smus 8y ago
  Hacker news jumps at any opportunity to write off ML
  - dang 8y ago
    Please don't post unsubstantive comments here.
- henning 8y ago
  There have been thousands of papers and many books written on the subject of using machine learning for time series forecasting and an unknown amount of money traded/invested with it. Most of them cast trading as a time series forecasting problem where performance is measured by mean square error. They usually then assume that would lead to profitable trading. You don't get to define what "modern machine learning" is any more than I do. The application of complex models to higher-dimensional domains have problems of their own. Both the successes and failures are worth writing and thinking about. The post is an important reminder about simplicity and complexity in modeling. That's why I submitted it.
  - stanfordkid 8y ago
    I don't understand why I'm being downvoted for this opinion -- the topic of overfitting gets beaten to death by the 3rd week of every intro AI/statistics/ML class out there. I didn't find it very worthy of the frontpage (or a blog post).

PLenz 8y ago

Yes, in a toy space ML replicates simplistic measures, there's only so much information to be had in such a space to start with. The power of these techniques is when we apply them to high-complexity or even unbounded-complexity areas.

candiodari 8y ago
And that you apply them to domains where the knowledge of the domain is basic -at best. It enables people who know one subject well -machine learning- to do speech recognition and weather forecasting well. Plus this article has a number of problems. It compares machine learning to MA(3). That it works pretty well totally ignores the selection of MA(3) vs MA(5). MA(100) is "common" indicator often talked up (why I do not know, except perhaps it protects you from really big errors, but other than that it's not very helpful). If you actually try to use the prediction of MA(3) you will find it is often very hard to use, because it's always just too late. I wonder if the machine learning version had the same problem, or not.
- afro88 8y ago
  > If you actually try to use the prediction of MA(3) you will find it is often very hard to use, because it's always just too late. I wonder if the machine learning version had the same problem, or not. Judging from the graph in the article, it looks like it doesn't. The model didn't reinvent the moving average, it invented it without the inherent delay. Though it does seem to diverge from real price over time.
- longerthoughts 8y ago
  >If you actually try to use the prediction of MA(3) you will find it is often very hard to use, because it's always just too late. I wonder if the machine learning version had the same problem, or not. Why does the sensitivity matter when we know that the accuracy was so similar? The point on sensitivity does make me wonder how an exponential moving average would've performed though.
  - candiodari 8y ago

TTPrograms 8y ago

"Is advanced machine learning a very complicated way of reproducing the results of trivial forecasting methods? In many cases it appears this is the case." If by "many cases" you mean "the one example I looked at which was carried out by an undergrad" than sure. That sort of phrasing really gives away the author's bias.

longerthoughts 8y ago
The author isn't arguing that ML isn't useful, they're simply pointing out that it isn't worthwhile for all problems and data sets.

m3kw9 8y ago

But the power comes when the model doesn’t overfit and can predict way more than your simple models

longerthoughts 8y ago

Why are so many people reading this as an attack on ML? The author is simply reminding people that ML isn't a universal remedy for uncertainty. "Stop writing ML models solely for the sake of writing ML models" is a very different message than "ML didn't improve upon MA(3) in this case so it's universally worthless".

iotasigma 8y ago

I understand that the general sentiment for this article seems to be dismissive but I want to share this awesome University of Vermont lecture series/course. It's related to forecasting mathematics and includes an introduction to fractal geometry which I haven't seen much of elsewhere. https://youtu.be/Lr3qrB6dPCM https://youtu.be/Lr3qrB6dPCM