5 ms·

I don't think I follow. These designs do come from first principles. In fact the reason we've had so much academic thought put into it is because 100 years ago

by hctaw 5y ago

I don't think I follow. These designs do come from first principles. In fact the reason we've had so much academic thought put into it is because 100 years ago we designed systems empirically and a few smart people went back to first principles to find the fundamental constraints on filter networks.

hcrisp 5y ago
That is how we did create such topologies. But if we had not, could they have been discovered through neural architecture search [1]? NAS has been used to find architectures that outperform those that were hand designed. And in RL , Google used a method inspired by NAS [2] to learn methods built by hand (i.e. TD learning, DQN). I know it's not the same but if you use a bit of speculative imagination you can translate what has been done to signal processing applications. [1] https://en.m.wikipedia.org/wiki/Neural_architecture_search https://en.m.wikipedia.org/wiki/Neural_architecture_search [2] https://ai.googleblog.com/2021/04/evolving-reinforcement-learning.html?m=1 https://ai.googleblog.com/2021/04/evolving-reinforcement-lea...