Ensembling is an old technique, used long before the wide/deep piece from Google. Using models pretained on ImageNet for transfer learning on new data - like y

Ensembling is an old technique, used long before the wide/deep piece from Google. Using models pretained on ImageNet for transfer learning on new data - like you said, you replace the last classifier layer, and potentially retrain the weights on some layers before it that are responsible for the higher level features. But of course you always test the results on your data specific training set