6 ms·

There seems to be a lot of angst in the responses about how deep nets et al "are too elegant; are too interpretable!". It's important to remember that the inte

There seems to be a lot of angst in the responses about how deep nets et al "are too elegant; are too interpretable!". It's important to remember that the intended product of science is not tools, it is understanding. Deep nets may produce very elegant and interpretable tools, from a very elegant and interpretable theory, but that is not what the scientists are looking for. They are generally creating and assessing theories in their subject domains, which are then evaluated by the tools they have built. Deep learning does do a great job of providing a baseline ("your theory must be this accurate to matter"), but it seems to do a much less great job at extracting new understanding.

backpropaganda 9y ago
Machine learning has departed from the venture of understanding-based science much before deep learning [1][2]. Thus, singling out deep learning reeks of motivated reasoning or just lack of awareness of the history of machine learning. Having said that, there still exists the lunatic fringe (it's a compliment) inside deep learning who continue to work on the task of generative modeling in the hope of "understanding the world" rather than "solving a task". Yoshua Bengio and Yann LeCun don't miss an opportunity to impress on the world how important unsupervised learning is. Physics too has abandoned understanding for predictive power. If you take philosophy of science seriously, you either have to pick "science is whatever the elite society believes in (Kuhn)" or "science gives us tools for prediction (Post-Kuhn)". Claiming that science helps in finding truth or understanding, is going to put you in a very indefensible slippery slope. "Understanding" is a subjective human concept, and not worth pursuing. Humans evolved to run from tigers, and struggle with harder tasks like understanding quantum mechanics or understanding the brain. The directionality of scientific progress as evidenced by QM is not "understanding", but tooling and predictive power. I hope Deep Learning is going to follow the path of predictive power, rather than regress into the pseudoscientific mess of elegance and understanding. [1]: http://www2.math.uu.se/~thulin/mm/breiman.pdf http://www2.math.uu.se/~thulin/mm/breiman.pdf [2]: http://norvig.com/chomsky.html http://norvig.com/chomsky.html
- bamboozled 9y ago
  "Understanding" is a subjective human concept, and not worth pursuing." Isn't human understanding of how our environment works precisely how you're able to sit in your chair and write comments on HN for others to read?
  - cercatrova 9y ago
    Sure but I might say that if I were to create a chair by generative methods, would I need to understand how it was created for it to be useful?
    - bamboozled 9y ago
      If you have ever crafted a chair, table, or anything else that is built by craftsperson, you will soon realise why such a profession exists, because it's very difficult. You might slap a few pieces of wood together and by brute force, create something like a chair and that might do, but it's very unlikely to be a good, safe, attractive, comfortable long lasting chair. Building quality furniture is complex and difficult work which takes a lot of skill, experience and understanding in order to do. I understand your point, I never said there is anything wrong with experimenting with NNs, I just don't agree with the OPs sentiment.
  - backpropaganda 9y ago
    No. Tools enabled it, not understanding.
    - j2kun 9y ago
      Give one specific example of a widespread modern invention that did not require any understanding to create. You personally can use a tool without understanding how it's made, but you seem to claim the tools are invented without anyone's understanding.
      - backpropaganda 9y ago

backpropaganda 9y ago

Machine learning has departed from the venture of understanding-based science much before deep learning [1][2]. Thus, singling out deep learning reeks of motivated reasoning or just lack of awareness of the history of machine learning. Having said that, there still exists the lunatic fringe (it's a compliment) inside deep learning who continue to work on the task of generative modeling in the hope of "understanding the world" rather than "solving a task". Yoshua Bengio and Yann LeCun don't miss an opportunity to impress on the world how important unsupervised learning is. Physics too has abandoned understanding for predictive power. If you take philosophy of science seriously, you either have to pick "science is whatever the elite society believes in (Kuhn)" or "science gives us tools for prediction (Post-Kuhn)". Claiming that science helps in finding truth or understanding, is going to put you in a very indefensible slippery slope. "Understanding" is a subjective human concept, and not worth pursuing. Humans evolved to run from tigers, and struggle with harder tasks like understanding quantum mechanics or understanding the brain. The directionality of scientific progress as evidenced by QM is not "understanding", but tooling and predictive power. I hope Deep Learning is going to follow the path of predictive power, rather than regress into the pseudoscientific mess of elegance and understanding. [1]: http://www2.math.uu.se/~thulin/mm/breiman.pdf http://www2.math.uu.se/~thulin/mm/breiman.pdf [2]: http://norvig.com/chomsky.html http://norvig.com/chomsky.html

bamboozled 9y ago
"Understanding" is a subjective human concept, and not worth pursuing." Isn't human understanding of how our environment works precisely how you're able to sit in your chair and write comments on HN for others to read?
- cercatrova 9y ago
  Sure but I might say that if I were to create a chair by generative methods, would I need to understand how it was created for it to be useful?
  - bamboozled 9y ago
    If you have ever crafted a chair, table, or anything else that is built by craftsperson, you will soon realise why such a profession exists, because it's very difficult. You might slap a few pieces of wood together and by brute force, create something like a chair and that might do, but it's very unlikely to be a good, safe, attractive, comfortable long lasting chair. Building quality furniture is complex and difficult work which takes a lot of skill, experience and understanding in order to do. I understand your point, I never said there is anything wrong with experimenting with NNs, I just don't agree with the OPs sentiment.
- backpropaganda 9y ago
  No. Tools enabled it, not understanding.
  - j2kun 9y ago
    Give one specific example of a widespread modern invention that did not require any understanding to create. You personally can use a tool without understanding how it's made, but you seem to claim the tools are invented without anyone's understanding.
    - backpropaganda 9y ago