4 ms·
Present day AI on the Deep Learning side is a lot like what you describe. We haven't really had the Newtonian foundations yet. The theoretical foundations are
by TimPC 9y ago
Present day AI on the Deep Learning side is a lot like what you describe. We haven't really had the Newtonian foundations yet. The theoretical foundations are quite limited because they are hard to figure out. But the techniques with less established theory work far better on most applications in AI. Redirecting work into areas of AI that have more solid theoretical foundations but worse application performance is not the way forward. I'm all for figuring out hard theoretical foundations but I'm strongly opposed to redirecting research funding to techniques that result in worse applications. I'd also argue modelling the physics isn't always the right approach: vocal tract modelling for speech is an interesting approach that produces much worse speech than state of the art synthesis techniques. It will probably continue to do so for a long time. For vocal tract modelling to produce better synthesis you'd need the physical model to be less lossy in all it's parameterizations and modelling simplifications than any statistical fitting of data. And you'd still need some statistical model of the choices the human makes in producing speech and you'd want that statistical model to work better than the neural network that takes on a larger portion of the problem and replaces the physical model of sound production.
- TimPC 9y agoI should point out in this case it's almost certainly a genuine call to research the foundations underlying the working techniques more as Duvenaud publishes research using mostly the techniques that work well on applications.
- joe_the_user 9y agoYou're right that the analogy doesn't imply that something analogous to physic is the answer. However, I would mention that there's a larger "overhead" than many realize to methods which work without the creator or the user understanding why. You have "racist" AI which don't undertstand that correlation may not be causation in questions like whether someone should be paroled or get a loan, you have the AIs subject to adversial attacks of various sorts, where not knowing why the AI works is also problematic, you have a situation where the target to match varies over time and so-forth. Which adds up to AI having more dimensions to it than simply "working well" and "working less well". Indeed, AI is effectively ad-hoc statistics with result derived heuristically. So in the process of "getting things right" exploring all sorts of things certainly sounds good, it seems like there's an "understanding gap" that needs to be closed and some broader model of what's happening would be useful but naturally there's no guarantee we can find one.