5 ms·
It is simple to say deep learning is based on "alchemy" or "engineering" or whatever it is that isn't strong theory. And it's reasonable to say deep learning ha
by joe_the_user 8y ago
It is simple to say deep learning is based on "alchemy" or "engineering" or whatever it is that isn't strong theory. And it's reasonable to say deep learning has a lot of mathematical and statistical intuitions but doesn't have a strong theory - maybe just doesn't yet have a strong theory or maybe can never get one.
So this is by now a standard argument. The standard answers I think have been:
1) Well, we are discovering that you don't need a strong theory for a truth-discovery machine. We've discovered experimental thinking construction.
2) It's true deep learning doesn't have a strong theory now but sooner or later it will get one.
3) This shows that instead we need theory X, usually by people who've been pursuing theory X all along. But I think Hinton at least has thrown a bunch of alternatives against the wall over the years.
(I could swear this has appeared here before but I can't find it).
- deleted 8y ago[deleted]
- AlexCoventry 8y agoI don't think it got as much traction, last time.
- bartread 8y agoWorth also bearing in mind that we've been here before in other fields. Alchemy ultimately became chemistry. Even in the Victorian era when, for fairly large swathes of the periodic table, and different types of compound, we already had a quite good experimental understanding of chemical reactions in terms of their constituent components and products, along with the conditions under which those reactions occur, we still didn't know the why. We didn't understand much about atoms or how they bond together, for example. The point is this: science can often take a long time to advance, and AI is still a very young field, with the first practical endeavours only dating back to the post-WWII period. Should we therefore be terribly surprised that ML seems a bit like alchemy? As an aside, another normal facet of scientific advancement is the vast quantity of naysayers encountered along the way. Haters gonna hate, I suppose. (But don't misunderstand me: whilst I'm not an ML fanboi, I recognise that advances come in fits and starts, dead-ends will be encountered, and overall it's going to take quite a long time and require a lot of hard work to get anywhere.) Final aside: this video has definitely been posted here before but I've also been unable to find it.
- joe_the_user 8y agoWorth also bearing in mind that we've been here before in other fields. Alchemy ultimately became chemistry. Indeed but that might be where the analogy breaks down. I think one say that we just don't know how far we can take "experimental computer science" - where the experimental part is making "random" or actually "seat-of-the-pants" programs and seeing what they do. This is simply new and one could create a physics on top of this particular kind of experimentation is yet to be seen.