3 ms·
TLDR: Because tree-based models don't just outperform deep learning, they totally outclass deep learning on simple data. But they don't scale with larger and m
by candiodari 3y ago
TLDR: Because tree-based models don't just outperform deep learning, they totally outclass deep learning on simple data.
But they don't scale with larger and more complex data. You cannot (realistically) make an LLM with XGBoost.
Kind of surprised how well Resnet and FT Transformer do though.
- nickpsecurity 3y agoI want to see more work combining them. Here’s an example I saw in one of the links in this thread: https://arxiv.org/abs/1806.06988 https://arxiv.org/abs/1806.06988 It combines NN’s with decision trees.
- huac 3y agothe famous FB ads paper (from 10 years ago!) combines decision trees with a logistic regression and shows a significant improvement: https://research.facebook.com/publications/practical-lessons-from-predicting-clicks-on-ads-at-facebook/ https://research.facebook.com/publications/practical-lessons... feel free to extend logistic regression to an MLP :)