jackylupino

3 ms·

1. 1.▲
   by jackylupino 5y ago

   If the paper would not have used this clickbait title, it would probably go unnoticed and noone would pay attention? Sad truth about sensation-oriented modern lives :(
2. 2.▲
   by jackylupino 5y ago

   It is true that MLPs are classic, but the regularizaton methods that apparently make a big empirical difference at this paper are new concepts (data augmentation, skip connections/residual blocks, dropout, batch norm, lookahead, stocha
3. 3.▲
   by jackylupino 5y ago

   Batch norm has an advantage for iterative methods on mini-batches, while XGB uses the full training set. Using batch norm on the full training set is equivalent to Z-normalizing the features, which has no effect at all for XGB as the scale
4. 4.▲
   by jackylupino 5y ago

   I agree with your point, e.g. data augmentation can be added, but thats pretty much it. All the other regularization techniques they use are neural network specific and cannot be applied to gradient-boosted trees. What I find particularly s
5. 5.▲
   by jackylupino 5y ago

   After reading the paper I believe they do both aspects that you mention: i) they give XGboost and all the baselines up to 4 CPU days of hyperparameter optimization time on 20 CPU core servers per dataset, same compute as the author's p