3 ms·
> Designing neural net architectures seems like exactly the sort of problem that neural nets ought to beat humans at handily. Is it? The “easy” tasks for neura
by dragonwriter 3y ago
> Designing neural net architectures seems like exactly the sort of problem that neural nets ought to beat humans at handily.
Is it? The “easy” tasks for neural nets to beat humans at are things that are harder for humans to do than for humans to provide volumes of high-quality training data by way of which neural nets can be trained.
It’s not clear to ne that designing neural nets really fits that.
- dsign 3y agoThe current crop of LLM are better aware of all the state of the art in NN architecture than yours-truly. They know things such as the number of layers used for X, Y and Z, the class of cost function which is popular with a specific architecture, dropout-vs-no-dropout, etc. That kind of knowledge still needs a human engineer to be leveraged, but even now it seems it's more because nobody has built a harness for the LLM to be able to put together neural network architectures. And, there are already automatic constraint checkers for NN architectures, so even if an LLM hallucinates something wrong, constraint checks could be used to automatically call its attention to some design flaws.