4 ms·
Neural Architecture Search with Reinforcement Learning
- jacquesm 10y agoAsk and ye shall receive... just a couple of hours ago I asked on twitter if such a thing has been done and I've been bombarded with prior attempts. Interesting how many different ways people have come to this idea. It's a powerful one: if architectures are where we can make a difference and architectures is a trial-and-error kind of thing at the moment that makes it a perfect candidate for automation.
- deepnotderp 10y agoYes, but most "architecture engineering" has already started to be phased out in deep learning research, with the exception of specifically tuned network types (i.e. for computational efficiency) or those with special layer types (i.e. separable convolutions, deconvolutions, etc.).
- oriel 10y agoCan you dump a list of the prior attempts you were sent?
- adw 10y agoHyperparameter optimization has been a thing forever.
- alexbeloi 10y agoI dig hyper-parameter optimization and architecture discovery but I'm always reminded of [this slide](https://image.slidesharecdn.com/k2jeffdean-160609173832/95/large-scale-deep-learning-with-tensorflow-46-638.jpg?cb=1465493958 https://image.slidesharecdn.com/k2jeffdean-160609173832/95/l...) and how these hp-tuning and architecture discovery tasks will always fall one tier above whatever your base optimization task is. The base tasks have to get faster for it to be practical for anyone without a server farm. I definitely think this is the future of ANNs, nets optimizing nets optimizing nets. Turtles all the way down.
- deepnotderp 10y agoThis has already been posted before, but the fact that DRL can do pretty good autoML is pretty useful!
- dankai 10y agoWow, this is amazing! I've been looking for work like this. At the same conference also this was published: HYPERNETWORKS [David Ha, et al] https://openreview.net/pdf?id=rkpACe1lx https://openreview.net/pdf?id=rkpACe1lx