3 ms·
A year ago, I did a review of the the GBT algorithms and settled on LightGBM for predictive performance and CPU time (not GPU). The benchmark differences are sm
by biomcgary 3y ago
A year ago, I did a review of the the GBT algorithms and settled on LightGBM for predictive performance and CPU time (not GPU). The benchmark differences are small enough that I'm reluctant to change, but I'd love to hear feedback.
Has anyone ever created a meta-ensemble model of several GBT algorithms?
- autokad 3y agoI worked with catboost, lightgbm, and xgboost when doing the zillow kaggle competition a long time ago, 2016 I think. I used what is called blending, where you give a weight to the models such as: .5 * xgboost + .25 * catboost + .25 * lightgbm
- knightoffaith 3y agoInteresting - did this work better than a single model? In general, do meta-ensembles work better? My sense was that just xgboost was the main winner of kaggle competitions.
- intelkishan 3y agoMeta-ensembles are the new trend for tabular data on Kaggle
- anmorgan24 3y agohttps://mlcontests.com/state-of-competitive-machine-learning-2023/ https://mlcontests.com/state-of-competitive-machine-learning...
- hcarlens 3y agoXGBoost, LightGBM, and Catboost are all used quite frequently in competitions. LightGBM is actually marginally more popular than the other two now, but it's pretty close. In the M5 forecasting competition a few years back, many of the top solutions used primarily LightGBM.
- shiandow 3y agoIf you're clever about how you blend models you can pretty much ensure that the performance of the (weighted) averaged model is strictly better than the (weighted) average performance of the models. And increasing the space of possible models pretty much guarantees the performance will improve, provided you can still find a good enough optimum.