3 ms·
I'm surprised the author didn't mention model variance reduction, which is what RFs were designed to do. For example, if we generate 100 data sets which are the
by chestervonwinch 11y ago
I'm surprised the author didn't mention model variance reduction, which is what RFs were designed to do. For example, if we generate 100 data sets which are the same except for noise, the 100 decision boundaries drawn by a classification tree will vary much more than the decision boundaries drawn by RFs on the same sets.
Here's what this means visually:
http://i.imgur.com/IjfXFkm.png http://i.imgur.com/IjfXFkm.png
There is just one of the 100 data sets generated shown.
- reagency 11y agoHmm, those CT boundaries aren't just high variance, they have extremely high error, which seems a more significant problem. They are linear!
- chestervonwinch 11y agoEach CT has zero error on its training set. The overall structure in each training set is the same (shown by the circles and triangles). Each CT, however, fits to the noise that is specific each training set. This causes the "linear" regions you mention. This is caused by how CTs recursively subdivide the feature space - these are essentially small subdivision areas fit to noise, which have extended beyond the class region to which they "should" belong.