3 ms·

I love both lightgbm and xgb. together they make a good ensamble. should be interesting how this one turns out.

• nerdponx 9y ago
  Interesting idea to ensemble them. The main difference is the way the trees are constructed, right? Do ensembling them is kind of like saying "I don't know which one is better, so screw it I will do both an average the results", right?
  • robrenaud 9y ago
    Lesser predictors can actually help in ensembles, so long as their errors aren't highly correlated with the better predictors.
  • autokad 9y ago
    if i am doing regressors, I use weights: like .7 xgb .3 lgb. I think its because they have their biases but the biasses are not completely correlated. This is one of the leading strategies in the kaggle competition for zillow logerror, check out this kernel: https://www.kaggle.com/aharless/xgb-w-o-outliers-lgb-with-outliers-combined https://www.kaggle.com/aharless/xgb-w-o-outliers-lgb-with-ou... When doing classification, I use several classifiers and get their scores rather than prediction via 20 fold xvalidation. I then use a random forest on top of the scores to get the final prediction. the random forest can figure out when to trust one classifier over another
  • s0ulmate 9y ago
    Hey! CatBoost team here. Yes, stacking different gradient boosting algorithms works well on practice. One example is a just finished kaggle-like competition: http://mlbootcamp.ru/round/12/sandbox/ http://mlbootcamp.ru/round/12/sandbox/ where mpershin stacked CatBoost with LGBM and got the 7th place. The kernel for this solution can be found in one of our tutorials: https://github.com/catboost/catboost/blob/master/catboost/tutorials/mlbootcamp_v_tutorial.ipynb https://github.com/catboost/catboost/blob/master/catboost/tu...