3 ms·
I can accept Newton's second law of motion and Einstein's general relativity, but so far I'm less accepting of deep learning for finding some coefficients for f
by graycat 10y ago
I can accept Newton's second law of motion and Einstein's general relativity, but so far I'm less accepting of deep learning for finding some coefficients for fitting some data or neural networks for fitting some sigmoid functions to some data. Apparently the sigmoid functions have some nice theoretical properties and can be useful in approximating some complicated data; e.g., IIRC D. Bertsekas at MIT used such fitting to approxmate the optimal value functions in stochastic optimal control. Or, have some big data and a lot of computing cycles? One way to soak them up is stochastic optimal control, possibly with some sigmoid fitting. To me, that's more promising for the future than what I've seen in the current AI based on Breiman, etc., and I'm a really big fan of Breiman.
What is going on in such cases -- learning, neural -- is at best some applied math, but the learning, neural, and intelligence are trying to suggest, hint, hint, that the math is somehow close to what intelligent humans with biological brains do.
The effects of the hints are strong enough to give lots of newsies another of their favorite means of getting eyeballs and clicks, that is, there are big threats, the sky is falling, nearly all humans will lose their jobs, and the robots might take over as in some movies.
Yes, somewhere in a buried paragraph there maybe an admission that the learning is likely nothing like what humans, kitty cats, puppy dogs, whales, or crows do, but that disclaimter, if want to call it that, is hardly a drop of water on the hype inferno.
At one time some IBM publicity got some housewives to believe that iBM's computers were "gigantic electronic human brains".
I know; I know; such hype is not the worst problem in the world and, really, is affecting me hardly at all. But when I was in an AI group at IBM's Watson lab, still I was embarrassed to say artificial intelligence. I knew, yes, that the meaning was supposed to be just a computer doing something that would require human intelligence if a human did it, e.g., play chess well, but that the approach of the AI chess, a big tree search with alpha-beta pruning, etc., was nothing like what humans did playing chess. But with this definition some 0-1 integer linear programming for scheduling at an airline should be far above merely intelligent since it is far above what an unaided human could do -- although humans invented the applied math that is 0-1 integer linear programming.
I call the crucial core technology for my startup not AI but just some applied math. The world is awash in good applied math -- statistical hypothesis testing, the fast Fourier transform, numerically exact matrix inversion (number theory techniques), Wiener filtering, deterministic optimal control, and hundreds more -- all without a single claim of anything intelligent in any of the books, papers, courses, etc.
- cr0sh 10y agoI am not an expert in this field. From what I currently understand in the CNN/deep learning arena, sigmoids are avoided in favor of RELU, due to a number of reasons, chiefly that of avoiding gradient fade during backprop. I'm surprised that you don't mention this (or don't seem to be aware of it), though? Is this intentional, or is there another reason?
- graycat 10y agoI mentioned fitting sigmoids as part of AI but did not intend that they were part of deep learning.
- Quarrelsome 10y agoOh you don't have to preach here, I am SOOOOOO converted. I spend way too much time pissing against the wind in reddit.com/r/futurology where it seems waitbutwhy is mandatory biblical reading. Then they pretend they like science at the same time :S. Still, my point is that I think IRRESPECTIVE of how we present it or what we do THEY will always jump to that conclusion. Remember that these are people that think vanilla programming is the work of a "computer" as opposed to it being an echo of the mind of a software engineer.