TL;DR: It depends on the number of subnetworks you search over and the cost of training each subnetwork. You could define a search space with a single DNN, and

TL;DR: It depends on the number of subnetworks you search over and the cost of training each subnetwork. You could define a search space with a single DNN, and give it one single iteration, and this would behave identically to the canned DNNEstimator. In a slightly more interesting case, suppose your search space consists of (say) 5 DNNs that cost you X each to train over (say) 1 epoch per iteration, and you train them over (say) 10 iterations, that would cost you around X x 5 x 10 = 50X. Do consider, though, that using AdaNet to explore, train, select, and ensemble these 50 DNNs is probably worth it considering how annoying it would be to do it without AdaNet :) Additionally, since AdaNet is implemented as a TensorFlow Estimator, it is easy to scale up the number of machines to speed up the training, if this is something you want.