3 ms·
This is basically what the field of neural architecture search tries to do. Here’s a good (somewhat technical) introduction: https://lilianweng.github.io/lil-lo
by jellyksong 5y ago
This is basically what the field of neural architecture search tries to do. Here’s a good (somewhat technical) introduction: https://lilianweng.github.io/lil-log/2020/08/06/neural-architecture-search.html https://lilianweng.github.io/lil-log/2020/08/06/neural-archi...
- IdiocyInAction 5y agoI think NAS is a bit higher level than what the OP had in mind - NAS isn't usually used to search for fundamental operations like self-attention or convolution. But I guess you could probably adapt it quite easily.
- jellyksong 5y agoI believe some algorithms, like AutoML-Zero, search on the level of mathematical operations.