3 ms·
We don't know which model architecture will work for which problem, and why it would/wouldn't. We experiment until we find something that works, and can sometim
by b3kart 6y ago
We don't know which model architecture will work for which problem, and why it would/wouldn't. We experiment until we find something that works, and can sometimes try to guess why it did. But none of this knowledge is formalized in a way that can reliably predict performance in future problems. We engineer solutions to problems, but don't build a rigorous body of knowledge to help us in future problems.
- glial 6y agoWell put. I would say that often even the problems aren’t particularly well defined.
- GregarianChild 6y agoThe problem you describe (model architecture will work for which problem) is not lacking rigour, but due to the Turing complete expressive power of neural networks. What you can do is use the existing rigour and derive stronger properties for restricted classes of NNs (just like you can prove stronger properties for simple subsystems of lambda-calculus) and that is an interesting field of study.
- eli_gottlieb 6y ago> The problem you describe (model architecture will work for which problem) is not lacking rigour, but due to the Turing complete expressive power of neural networks. Non-recurrent neural networks are not Turing-complete in any sense.