Great question! In the simplest case, AdaNet allows you to ensemble independent subnetworks from a linear model to user-defined DenseNet/AmoebaNet-style network

Great question! In the simplest case, AdaNet allows you to ensemble independent subnetworks from a linear model to user-defined DenseNet/AmoebaNet-style networks. But more interesting is sharing information (tensor outputs or which hyperparameters worked best) between iterations so that AdaNet can do neural architecture search for you. Users can define their own adanet.subnetwork.Generator in order to specify how to adapt training across iterations. Out of the box, the meta-strategy is little more than simple-user defined heuristics (e.g., “if the the deepest candidate subnetwork performed best, try subnetworks that are one layer deeper than that”). However, the AdaNet framework is flexible enough to support smarter strategies as you mentioned, and abstracts away the complexities of distributed training (Estimator), evaluation (TensorBoard), and serving (tf.SavedModel).