4 ms·
I actually think that random forests are relatively difficult to tune in comparison to some Boosting methods for example. The typical parameters are the follow
by vladoh 13y ago
I actually think that random forests are relatively difficult to tune in comparison to some Boosting methods for example.
The typical parameters are the following:
- Number of trees
- Percentage of data used to train each tree
- Maximal depth of the tree
- Minimum information gain (although this can usually be set to 0 and use only the depth)
- Minimum sample size (same as the minimal information gain)
- Number of thresholds to try for continuous data
Tuning is also difficult because of the non deterministic nature of the algorithm. If you compare to sets of parameters it requires more evaluation in order to be sure if one set is better because of the better choice of the parameters or because the algorithm chose the right data samples. This effect decreases with the number of trees, but the training time is increased in this case.
I think random forests are really good for very large datasets, but in my experience for smaller datasets boosting (for example JointBoost or GentleBoost) can give better results.
- taliesinb 13y agoSpeaking of bagging: Geoff Hinton talks a bit about model averaging and has a very stimulating pair of hypotheses related to this, which are: a) genetic recombination in the form of sex might be "bagging across genes" that prevents tight co-adaptation (= overfitting on the evolutionary timescale) b) the brain uses noisy discrete firing instead of continuous communication because it allows for "bagging across network topologies". The bulk of his talk is about how dropout in neural nets can be used to accomplish some of the same performance benefits that model averaging gives on other algorithms (at a much smaller relative computational cost). Here's the talk: https://www.youtube.com/watch?v=DleXA5ADG78 https://www.youtube.com/watch?v=DleXA5ADG78