I agree with your point, e.g. data augmentation can be added, but thats pretty much it. All the other regularization techniques they use are neural network spec

I agree with your point, e.g. data augmentation can be added, but thats pretty much it. All the other regularization techniques they use are neural network specific and cannot be applied to gradient-boosted trees. What I find particularly striking at this paper is that their method trains a single neural network which outperforms an ensemble of decision trees (XGBoost). Asking for perfect apple-to-apple comparisons means also comparing an ensemble of the MLPs vs. XGBoost. In this context, at least the message here is that XGBoost and/or other gradient-boosted methods are not anymore a silver bullet for tabular datasets. Boosting for trees was great in reducing both bias and variance, but apparently neural networks can achieve the same effect with a high capacity (low bias) and a mix of modern regularization techniques (low variance).