4 ms·
> Practical use: Once you have GNNs that predict the trained weights of headless models for vision, NLP, and other domains, you can test new architectures in al
by webmaven 5y ago
> Practical use: Once you have GNNs that predict the trained weights of headless models for vision, NLP, and other domains, you can test new architectures in all those domains much faster with fewer compute resources.
Aha. Using this as an accelerator for Neural Architecture Search would indeed be very neat.
- hwers 5y agoYou still need to train 1 million architectures though before you can start the search (dataset inflexible)… I suppose there’s some value in possibly finding radically different networks (though I wonder if it’d fare well in such outlier regions). Perhaps if/since the GNN model is also differentiable you could invert it to maximize some compute/quality trade off point.
- joshuamorton 5y agoBut like it already exists. That's a sunk cost. Presuming it actually works and gives useful weights, if it lowers training time of novel models by 10% or something, that's dollars saved in the long run.
- deleted 5y ago[deleted]