4 ms·

As interesting as I find the current state of deep learning to be, there is something about random forests that I can't help but find much more cool. Probably t

As interesting as I find the current state of deep learning to be, there is something about random forests that I can't help but find much more cool. Probably the amazing out-of-box performance.

petters 9y ago
Yes, they have very few knobs to turn, which is very attractive.
platz 9y ago
also the model is analyzable so as to determine the variables which are contributing the most.
- codewithcheese 9y ago
  You might be interested in this approach to explaining the predictions for any classifier. https://github.com/marcotcr/lime https://github.com/marcotcr/lime
- nerdponx 9y ago
  For some very nice Random Forest visualizations, check out the R package "forestFloor" [0]. I also once started implementing a R package for "partial dependence plots" [1][2], which are popularly associated with Random Forests but aren't specific to them. [0]: https://CRAN.R-project.org/package=forestFloor https://CRAN.R-project.org/package=forestFloor [1]: http://scikit-learn.org/stable/auto_examples/ensemble/plot_partial_dependence.html http://scikit-learn.org/stable/auto_examples/ensemble/plot_p... [2]: https://github.com/gwerbin/statsplots/blob/master/R/partialplot.R https://github.com/gwerbin/statsplots/blob/master/R/partialp...
nerdponx 9y ago
You might also be interested in algorithms like Adaboost [0] and its successors, including the various "gradient boosting" algorithms like XGBoost, LightGBM, and the newly-open-sourced Catboost. 0: https://jeremykun.com/2015/05/18/boosting-census/ https://jeremykun.com/2015/05/18/boosting-census/
- shoo 9y ago
  a great paper on this is Friedman's "gradient boosting machine" paper, where he shows how the boosting idea can be generalised to support a range of different loss functions and underlying approximation schemes (especially trees). "Greedy function approximation: a gradient boosting machine" - JH Friedman

As interesting as I find the current state of deep learning to be, there is something about random forests that I can't help but find much more cool. Probably the amazing out-of-box performance.

petters 9y ago

Yes, they have very few knobs to turn, which is very attractive.

platz 9y ago

also the model is analyzable so as to determine the variables which are contributing the most.

codewithcheese 9y ago
You might be interested in this approach to explaining the predictions for any classifier. https://github.com/marcotcr/lime https://github.com/marcotcr/lime
nerdponx 9y ago
For some very nice Random Forest visualizations, check out the R package "forestFloor" [0]. I also once started implementing a R package for "partial dependence plots" [1][2], which are popularly associated with Random Forests but aren't specific to them. [0]: https://CRAN.R-project.org/package=forestFloor https://CRAN.R-project.org/package=forestFloor [1]: http://scikit-learn.org/stable/auto_examples/ensemble/plot_partial_dependence.html http://scikit-learn.org/stable/auto_examples/ensemble/plot_p... [2]: https://github.com/gwerbin/statsplots/blob/master/R/partialplot.R https://github.com/gwerbin/statsplots/blob/master/R/partialp...

nerdponx 9y ago

You might also be interested in algorithms like Adaboost [0] and its successors, including the various "gradient boosting" algorithms like XGBoost, LightGBM, and the newly-open-sourced Catboost. 0: https://jeremykun.com/2015/05/18/boosting-census/ https://jeremykun.com/2015/05/18/boosting-census/

shoo 9y ago
a great paper on this is Friedman's "gradient boosting machine" paper, where he shows how the boosting idea can be generalised to support a range of different loss functions and underlying approximation schemes (especially trees). "Greedy function approximation: a gradient boosting machine" - JH Friedman