For many and perhaps most state-of-the-art models, the answer to "why?" is "because it works." As a practitioner, I would say Deep Learning is a trade. The mor

For many and perhaps most state-of-the-art models, the answer to "why?" is "because it works." As a practitioner, I would say Deep Learning is a trade. The more you work with these models, the more you develop intuitions and habits for things that work and things that don't, like a sculptor who learns to craft beautiful or functional objects by chiseling stone. ("Why did you hit the marble that way?" "Because it works.") If we want to be more generous, we can call Deep Learning an experimental science, because some researchers working with deep models are truly doing methodical, tedious, experimental work and documenting it for posterity. They and everyone else, including me, hope that we will eventually be able to answer the "whys." But there's no guarantee, a priori, that we will find satisfactory answers. Quoting Rich Sutton: "the actual contents of minds are tremendously, irredeemably complex; we should stop trying to find simple ways to think about the contents of minds, such as simple ways to think about space, objects, multiple agents, or symmetries. All these are part of the arbitrary, intrinsically-complex, outside world. They are not what should be built in, as their complexity is endless; instead we should build in only the meta-methods that can find and capture this arbitrary complexity. Essential to these methods is that they can find good approximations, but the search for them should be by our methods, not by us. We want AI agents that can discover like we can, not which contain what we have discovered. Building in our discoveries only makes it harder to see how the discovering process can be done." (Source: http://incompleteideas.net/IncIdeas/BitterLesson.html http://incompleteideas.net/IncIdeas/BitterLesson.html) Quoting Geoff Hinton: "One place where I do have technical expertise that’s relevant is [whether] regulators should insist that you can explain how your AI system works. I think that would be a complete disaster. People can’t explain how they work, for most of the things they do. When you hire somebody, the decision is based on all sorts of things you can quantify, and then all sorts of gut feelings. People have no idea how they do that. If you ask them to explain their decision, you are forcing them to make up a story. Neural nets have a similar problem. When you train a neural net, it will learn a billion numbers that represent the knowledge it has extracted from the training data. If you put in an image, out comes the right decision, say, whether this was a pedestrian or not. But if you ask “Why did it think that?” well if there were any simple rules for deciding whether an image contains a pedestrian or not, it would have been a solved problem ages ago." (Source: https://www.wired.com/story/googles-ai-guru-computers-think-more-like-brains https://www.wired.com/story/googles-ai-guru-computers-think-...)