I'm not sure this is surprising. Say you were to glue together 10 datasets with the same 10 explanatory features and 1 response feature, but distributed very di

I'm not sure this is surprising. Say you were to glue together 10 datasets with the same 10 explanatory features and 1 response feature, but distributed very differently to each other. This would be no problem for tree based model because they'll conditionalise indefinitely to get a good fit. If the number of records is relatively small (say 10k) the dataset will be much too scarce for an NN to learn these discontinuities -- its like it has 1000 records per segment. Similarly, tabular data is often of this nature. Its not i.i.d, it tends to cluster.