5 ms·

RandomForest and other decision tree methods can actually handle missing data very well by treating an individual cell in the data matrix as missing, rather tha

RandomForest and other decision tree methods can actually handle missing data very well by treating an individual cell in the data matrix as missing, rather than discarding the entire row. So the assertion that "All algorithms should operate only on data vectors and on frequency weights — they should have no knowledge of missing-ness." is false - there are a lot of other fruitful ways to handle missing data, such as using indicator variables, imputation, etc. - see http://www.amazon.com/Statistical-Analysis-Missing-Roderick-Little/dp/0471183865/ref=sr_1_13?s=books&ie=UTF8&qid=1419026924&sr=1-13&keywords=missing+statistics http://www.amazon.com/Statistical-Analysis-Missing-Roderick-...

shoo 12y ago
If anyone is particularly interested, there are notes on the original RandomForest implementation, by Breiman and Cutler, which describe how missing values are handled (as well as a great deal of other interesting tricks!): https://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm#missing1 https://www.stat.berkeley.edu/~breiman/RandomForests/cc_home...
- micro_cam 12y ago
  There are some other ways an rf can be made to handle missing values including splitting the values onto a third branch at each node (which can be useful if the fact the value is missing may be informative), just leaving the missing values out at each split and applying a bias correction (as was common with pre rf decision trees), or doing a local imputation within the in branch cases at each node.
IndianAstronaut 12y ago
>RandomForest and other decision tree methods can actually handle missing data very well by treating an individual cell in the data matrix as missing This is why I tend to favor these over regression or other methods. Real data is going to have lots of missing values, sometimes too many for imputation to even work. Whittling down your data to only those which have all predictor fields filled in can drastically shrink down your data set.

shoo 12y ago

If anyone is particularly interested, there are notes on the original RandomForest implementation, by Breiman and Cutler, which describe how missing values are handled (as well as a great deal of other interesting tricks!): https://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm#missing1 https://www.stat.berkeley.edu/~breiman/RandomForests/cc_home...

micro_cam 12y ago
There are some other ways an rf can be made to handle missing values including splitting the values onto a third branch at each node (which can be useful if the fact the value is missing may be informative), just leaving the missing values out at each split and applying a bias correction (as was common with pre rf decision trees), or doing a local imputation within the in branch cases at each node.

IndianAstronaut 12y ago

>RandomForest and other decision tree methods can actually handle missing data very well by treating an individual cell in the data matrix as missing This is why I tend to favor these over regression or other methods. Real data is going to have lots of missing values, sometimes too many for imputation to even work. Whittling down your data to only those which have all predictor fields filled in can drastically shrink down your data set.