4 ms·

Author here with several of the core team; we can answer some questions here. We also have a post on r/machinelearning for further discussion: https://www.reddi

• bitL 8y ago
  I haven't read it in detail so my apologies if you explain this in the notebooks; can AdaNet handle blocks with variable-length skip connections (like DenseNet), or even come up with AmoebaNet-style models on its own? What is the meta-strategy guiding the hyperparameter/architecture selection process (grid search/Bayesian/etc.)? Thanks!
  • cweill 8y ago
    Great question! In the simplest case, AdaNet allows you to ensemble independent subnetworks from a linear model to user-defined DenseNet/AmoebaNet-style networks. But more interesting is sharing information (tensor outputs or which hyperparameters worked best) between iterations so that AdaNet can do neural architecture search for you. Users can define their own adanet.subnetwork.Generator in order to specify how to adapt training across iterations. Out of the box, the meta-strategy is little more than simple-user defined heuristics (e.g., “if the the deepest candidate subnetwork performed best, try subnetworks that are one layer deeper than that”). However, the AdaNet framework is flexible enough to support smarter strategies as you mentioned, and abstracts away the complexities of distributed training (Estimator), evaluation (TensorBoard), and serving (tf.SavedModel).