I think there is definitely a path towards making neural nets more of a science. Trouble is, it's just so easy to get results quickly and publish without too mu

I think there is definitely a path towards making neural nets more of a science. Trouble is, it's just so easy to get results quickly and publish without too much deep understanding. The meme is that neural nets work only in practice, not theory. I do not think we should dismiss it though, because people are working hard to bring the theory up to speed. Neural nets are also showing a lot of promise for symbolic tasks like automated theorem proving. So I would predict that in time they eventually become a standard technique in what would otherwise be the domain of older techniques.