You could think of this as being hyperparameter tuning, but that would be simplistic. Previous approaches have tried to hold most of architecture constant and

You could think of this as being hyperparameter tuning, but that would be simplistic. Previous approaches have tried to hold most of architecture constant and then scientifically report the results of tweaking a particular variable for verifiable results. Quoc has been working with evolutionary approaches to performing these sort of experiments with large clusters. Most of the results so far have been technically superior, but at the cost of complexity/understandability. Here though, is the logical conclusion of this approach done at scale: using evolutionary approaches to explore a large swath of architecture space, finding an entirely new category of convolutional networks, then working backwards to a formal/clean architecture and paper.