3 ms·
GitHub Repo: Please see the results folder Results folder: Here's some undocumented parquet files Definitely feels like they're hiding the ball lol. If they
by Eridrus 3mo ago
GitHub Repo: Please see the results folder
Results folder: Here's some undocumented parquet files
Definitely feels like they're hiding the ball lol.
If they had good benchmarks they'd talk about them.
Not comparing to tuned xgboost is also a warning sign.
- nok22kon 3mo agowouldn't xgboost be covered under autogluon? not perfect, but not missing either
- Eridrus 3mo agoHonestly, I don't really know AutoGluon, if this does xgboost tuning that's good. I do still think ELO scores are still a way to obscure results though. For all we know this is like 0.1% better than a "normal" approach on like 70% of tasks and a tire fire on others.
- nok22kon 3mo agoAutoGluon does their own benchmarking, the default is Elo, but you can switch to other metrics: https://huggingface.co/spaces/TabArena/leaderboard https://huggingface.co/spaces/TabArena/leaderboard xgboost is listed, they say "tuned" but who knows what that means. and its below CatBoost and LightGBM as an aggregator of models (trees, neural networks, clustering, ...) AutoGluon doesn't really have a dog in this fight
- Eridrus 3mo agoThanks for the pointer, though I can't seem to find TabFM and AutoGluon at that link?