4 ms·
My Observations: * Instance-based methods can adapt to any kind of pattern if you have enough data. This is a well-known result in machine learning -- the inte
by apu 17y ago
My Observations:
* Instance-based methods can adapt to any kind of pattern if you have enough data. This is a well-known result in machine learning -- the interesting question is simply how much data do you need?
* It's quite remarkable how well RBF SVMs do in almost all cases. They even outperformed a 2nd degree polynomial kernel SVM on a 2nd degree polynomial!
* Logistic regression is pretty terrible for anything except "easy" data -- simple linear data.
* Random forests are sometimes ok, but tend to prefer axis-aligned data (at least in his formulation).
* Meta-point: all the tests shown are with "clean" data...i.e., there are no "wrong" training examples. This is unrealistic, and in practice makes a HUGE difference for some of these methods. E.g., a lot of the rule-based methods get demolished by even a little bit of wrong data. In contrast, SVMs have a slack variable that can tolerate some amount of noise, and would probably shine even more on such data.
- greendestiny 17y agoI think its interesting how well the instance based methods perform at comparative numbers of training samples to the SVM-RBF methods. I'd certainly choose instance based methods for tasks with similar data to these tests.
- clay 17y agoDoing predictions with instance based methods can be really expensive.
- eob 17y agoShouldn't an RBF kernel be expected to perform better than a 2nd degree poly kernel since, technically, it is infinite-dimensional? (I.e., the radial basis function can be expanded into an infinite-degree taylor series) I think (not a ML expert by any means) that RBF should probably outperform most fixed-integer polynomial kernels, but it does so at the cost of introducing more structural risk into the model.
- alextp 17y agoIt does not really go like that. The higher the dimension (actually VC dimension, but polynomial dimension is a similar concept) the more points you would theoretically need to guarantee that the classifier is not overfitting (ie, representing the noise in the data set).