3 ms·

When applying CS/ML to finance, methodology is absolutely crucial. Yet most work - even work done by quants in major sell-side banks - pays little attention to

by a008t 8y ago

When applying CS/ML to finance, methodology is absolutely crucial. Yet most work - even work done by quants in major sell-side banks - pays little attention to methodology; it is often naively assumed that the methodology applied to classifying cat images can be directly transferred into the domain of finance. Here is an article that explains the problem well: http://zacharydavid.com/2017/08/06/fitting-to-noise-or-nothing-at-all-machine-learning-in-markets/ http://zacharydavid.com/2017/08/06/fitting-to-noise-or-nothi...

zmk_ 8y ago
But that's because it is being applied by people who come from classifying cat images and have little domain knowledge.
nonbel 8y ago
>"When applying CS/ML to finance, methodology is absolutely crucial." When wouldnt it be?
- a008t 8y ago
  In other areas, we can often assume that each example in our dataset is an independent sample from some underlying distribution. We can also assume that the distribution stays constant. Furthermore, we have fairly high-accuracy ground truth labels. Also, we know that everything we need to classify a sample is in the data we are given. We may also have more data to play with than we really need. Take handwritten digit recognition task like MNIST, for example - the way we write the number '5' does not change much, and the distribution of the different ways people write that number stays pretty much the same over time. The labels we are given are very accurate and everything we need to classify the image is in the data. All these assumptions mean that picking the right methodology is fairly easy - anyone can read a short tutorial and get it right. None of the assumptions I listed hold in the world of finance. There is no standard methodology that everyone can agree should be followed. You really have to be very, very careful in order to produce useful results.
  - nonbel 8y ago
    It sounds like you just need to use a different methodology, not that methodology is unimportant for whatever ml 101 examples that became popular like MNIST, etc.
    - a008t 8y ago
      Yes, but I also think it's a lot easier to get it wrong in finance without even realizing it than in many other domains. Relatively more difficult to screw up the methodology on MNIST. Many mistakenly assume that the methodology from MNIST transfers directly to finance, which makes the problem worse. ALl of that means that when talking about applying ML to finance, discussing the methodology in detail is a must. If it is not talked about or only briefly mentioned, from my experience that usually means that the methodology used is rubbish. Whereas you don't need to focus as much on the methodology when talking about MNIST-like problems - one can usually assume a reasonable one is used.

by a008t 8y ago

zmk_ 8y ago

But that's because it is being applied by people who come from classifying cat images and have little domain knowledge.

nonbel 8y ago

>"When applying CS/ML to finance, methodology is absolutely crucial." When wouldnt it be?

a008t 8y ago
In other areas, we can often assume that each example in our dataset is an independent sample from some underlying distribution. We can also assume that the distribution stays constant. Furthermore, we have fairly high-accuracy ground truth labels. Also, we know that everything we need to classify a sample is in the data we are given. We may also have more data to play with than we really need. Take handwritten digit recognition task like MNIST, for example - the way we write the number '5' does not change much, and the distribution of the different ways people write that number stays pretty much the same over time. The labels we are given are very accurate and everything we need to classify the image is in the data. All these assumptions mean that picking the right methodology is fairly easy - anyone can read a short tutorial and get it right. None of the assumptions I listed hold in the world of finance. There is no standard methodology that everyone can agree should be followed. You really have to be very, very careful in order to produce useful results.
- nonbel 8y ago
  It sounds like you just need to use a different methodology, not that methodology is unimportant for whatever ml 101 examples that became popular like MNIST, etc.
  - a008t 8y ago
    Yes, but I also think it's a lot easier to get it wrong in finance without even realizing it than in many other domains. Relatively more difficult to screw up the methodology on MNIST. Many mistakenly assume that the methodology from MNIST transfers directly to finance, which makes the problem worse. ALl of that means that when talking about applying ML to finance, discussing the methodology in detail is a must. If it is not talked about or only briefly mentioned, from my experience that usually means that the methodology used is rubbish. Whereas you don't need to focus as much on the methodology when talking about MNIST-like problems - one can usually assume a reasonable one is used.