Agreed, isolating the contribution of the "model" itself can be borderline impossible, and the details of model engineering have a considerable impact on perfor

Agreed, isolating the contribution of the "model" itself can be borderline impossible, and the details of model engineering have a considerable impact on performance. That's why I like Kaggle competitions: most approaches tried on a given challenge will likely be optimized to their very limits, so that it's the nature of the different approaches that ends up making the difference.