3 ms·

“Machine learning algorithms will always identify a pattern, even if there is none” Is this partly a problem with interpretation? Let's say I do a binary (supe

by geebee 8y ago

“Machine learning algorithms will always identify a pattern, even if there is none” Is this partly a problem with interpretation? Let's say I do a binary (supervised) classification with an algorithm that is also capable of assessing probabilities. If I generate a data set consisting of a randomized bag of words, and randomly assign them to 0 and 1 categories, and run it through a supervised ML classifier, then yeah, everything in the test set will get assigned to something. But if you look at the probability estimates resulting from the ML, you'd almost certainly see something that indicates a high degree of randomness in the assignments (various techniques such as cross validation, or probabilities that indicate a high degree of uncertainty for almost all of the predictions). I'm not sure this is a problem with the algorithm itself, because the output from many of these algorithms does indicate low predictive value.

sdenton4 8y ago
Check out this classic paper on deep learning with randomized labels: https://arxiv.org/abs/1611.03530 https://arxiv.org/abs/1611.03530 Spoiler: the neutral net thinks it's doing a really good job!
- geebee 8y ago
  Thank you for the link. I'll read this paper. I'm hoping to reply but the thread may be stale by the time I do. Right now, my thoughts are: if it is easily fitting, what are the assignment probabilities? Are we getting 90%+, or is it fitting easily, but to much lower probabilities. Also, is there a big difference between neural nets and other algorithms like RF? The paper certainly does appear to address the question of categorizing completely randomized input: From the abstract "...our experiments establish that state-of-the-art convolutional networks for image classification trained with stochastic gradient methods easily fit a ran- dom labeling of the training data. This phenomenon is qualitatively unaffected by explicit regularization, and occurs even if we replace the true images by com- pletely unstructured random noise.
  - geebee 8y ago
    I'm going off a first pass through the paper, but it appears that what this paper shows is that the training error can be 0 on an entirely randomized data set, but the generalization error - the difference between the error on the test set and the training set, does increase dramatically as label corruption increases. My understanding is that cross validation does multiple combinations of splitting the input data into test and training sets... so if cross validation measures the generalization error, wouldn't this catch the low predictive value resulting from randomization of labels or input? I'm not saying the paper doesn't have value, but I think it's more about the fact that neural nets can obtain a training error of zero on randomized data, not a testing error (or generalization error, which represents the difference between training error and testing error, as far as I can tell). To be clear, I'm not an expert, and this is just what I gleaned from a first pass over the paper.
    - sdenton4 8y ago
      All true. The interesting thing here is that the neural network has /no idea/ that it sucks at generalization, though. Yes, we can do extra work to calibrate outputs, but it would be much better to have some idea of uncertainty from the network itself. (Added as edit) also keep in mind that datasets themselves often fail to generalize - overriding to a particular set makes for domain error when moving to slightly different data. Cross validation won't help wit that, but more "self aware" algorithms might.
      - geebee 8y ago
        But... isn't that the entire point of splitting your initial training data into a training set and a separate testing set? Why is it better to have an idea of uncertainty from the model itself when you can get the generalization error through cross validation, or by setting aside a testing set? It's interesting to see that a neural net will reach a training error of zero on randomized data, and it's a worthwhile contribution to the literature to demonstrate this, test it, and measure it... but the outcome here doesn't surprise me. From experience I know that random forests will also show nearly 100% accuracy on a training set but show far lower accuracy for a testing set, so while I think it's great to measure it, the conclusion in this paper is not surprising. In no way is that a knock on the paper, people weren't surprised that Fermat's last theorem turned out to be true, but that doesn't make the proof any less of an accomplishment!
yters 8y ago
That's because ML is given a model to fit to the data, so it'll find the best fit, even if the model doesn't represent the data.
- geebee 8y ago
  Well, right. A binary classification system must assign a 0 or 1. But cross validation or other methods may reveal that it isn't a good fit, just a fit. As mlthoughts pointed out in a different comment, any kind of regression technique faces issues about goodness of fit. The thing is, there are techniques to show you that the fit isn't very good. A simple linear regression will fit randomized noise, but there are outputs that can show you that the fit isn't good and the regression may not be reliable. The question I have here is whether ML techniques are failing in a different way, that it is fitting to randomized noise while appearing by various tests to be a very strong fit. If they're failing the same way that regression would (i.e.., someone applies it and fails to do basic tests for goodness of fit), that's a problem I suppose, but is it really a unique failing of ML or neural nets? It sounds like more like a standard misapplication of predictive modeling...

by geebee 8y ago

sdenton4 8y ago

Check out this classic paper on deep learning with randomized labels: https://arxiv.org/abs/1611.03530 https://arxiv.org/abs/1611.03530 Spoiler: the neutral net thinks it's doing a really good job!

geebee 8y ago
Thank you for the link. I'll read this paper. I'm hoping to reply but the thread may be stale by the time I do. Right now, my thoughts are: if it is easily fitting, what are the assignment probabilities? Are we getting 90%+, or is it fitting easily, but to much lower probabilities. Also, is there a big difference between neural nets and other algorithms like RF? The paper certainly does appear to address the question of categorizing completely randomized input: From the abstract "...our experiments establish that state-of-the-art convolutional networks for image classification trained with stochastic gradient methods easily fit a ran- dom labeling of the training data. This phenomenon is qualitatively unaffected by explicit regularization, and occurs even if we replace the true images by com- pletely unstructured random noise.
- geebee 8y ago
  I'm going off a first pass through the paper, but it appears that what this paper shows is that the training error can be 0 on an entirely randomized data set, but the generalization error - the difference between the error on the test set and the training set, does increase dramatically as label corruption increases. My understanding is that cross validation does multiple combinations of splitting the input data into test and training sets... so if cross validation measures the generalization error, wouldn't this catch the low predictive value resulting from randomization of labels or input? I'm not saying the paper doesn't have value, but I think it's more about the fact that neural nets can obtain a training error of zero on randomized data, not a testing error (or generalization error, which represents the difference between training error and testing error, as far as I can tell). To be clear, I'm not an expert, and this is just what I gleaned from a first pass over the paper.
  - sdenton4 8y ago
    All true. The interesting thing here is that the neural network has /no idea/ that it sucks at generalization, though. Yes, we can do extra work to calibrate outputs, but it would be much better to have some idea of uncertainty from the network itself. (Added as edit) also keep in mind that datasets themselves often fail to generalize - overriding to a particular set makes for domain error when moving to slightly different data. Cross validation won't help wit that, but more "self aware" algorithms might.
    - geebee 8y ago
      But... isn't that the entire point of splitting your initial training data into a training set and a separate testing set? Why is it better to have an idea of uncertainty from the model itself when you can get the generalization error through cross validation, or by setting aside a testing set? It's interesting to see that a neural net will reach a training error of zero on randomized data, and it's a worthwhile contribution to the literature to demonstrate this, test it, and measure it... but the outcome here doesn't surprise me. From experience I know that random forests will also show nearly 100% accuracy on a training set but show far lower accuracy for a testing set, so while I think it's great to measure it, the conclusion in this paper is not surprising. In no way is that a knock on the paper, people weren't surprised that Fermat's last theorem turned out to be true, but that doesn't make the proof any less of an accomplishment!

yters 8y ago

That's because ML is given a model to fit to the data, so it'll find the best fit, even if the model doesn't represent the data.

geebee 8y ago
Well, right. A binary classification system must assign a 0 or 1. But cross validation or other methods may reveal that it isn't a good fit, just a fit. As mlthoughts pointed out in a different comment, any kind of regression technique faces issues about goodness of fit. The thing is, there are techniques to show you that the fit isn't very good. A simple linear regression will fit randomized noise, but there are outputs that can show you that the fit isn't good and the regression may not be reliable. The question I have here is whether ML techniques are failing in a different way, that it is fitting to randomized noise while appearing by various tests to be a very strong fit. If they're failing the same way that regression would (i.e.., someone applies it and fails to do basic tests for goodness of fit), that's a problem I suppose, but is it really a unique failing of ML or neural nets? It sounds like more like a standard misapplication of predictive modeling...