4 ms·
The answer for most of these problems where humans say "humans do X and X is difficult because ..." is either : 1) it's not complex at all. 2) same as 1, with
by iofj 11y ago
The answer for most of these problems where humans say "humans do X and X is difficult because ..." is either :
1) it's not complex at all.
2) same as 1, with "in 99.99...% of the cases" appended to it.
Most of the problems in AI are a scaling problem. The reason you're not yet seeing androids taking over the economy are twofold:
Portable energy. Our best technologies are not the equal of the human body when it comes to how much power can be stored. But they are constantly improving, even if we'll need another tenfold increase in energy density (less if we make mobile robots gasoline powered, which is impractical for other reasons, more if do the battery + electrical motors thing everyone wants). This means humans are cheaper and easier for a lot of tasks.
Control. The human body (not including face and face-adjecent muscles) has about 300 actuators. That means controlling a human body means controlling 300 individual motors at the same time in a useful way (and every motor affects every other motor. Moving your hand forward means adjusting the power your little toe is applying to the ground to maintain balance). The state of the art is maybe 10-dimensional control (10 interdependent actuators), which can be increased to maybe 20 if the problem can be split into subproblems (e.g. cooperating robots, or parts of the robot that are attached, so imbalance cannot occur). In some ways every extra dimension adds an order of magnitude to the complexity, so it's not like we'll get there in 10 years. But in a century ... probably.
The thing about these problems is that they are problems of degree. Just like today's AI algorithms were known in the 1960's. So why didn't we have live speech transcription in the 1960s ? We all know the answer : processing power was 50 orders of magnitude less than we needed. It was a problem of degree. We knew the problem, and had at least a good guess at what to do, but couldn't contemplate that actually acting on those ideas would make any serious progress due to the limitations of processing speeds at those times. If you wanted a 1960s computer to do a 50x50 matrix multiplication, you'd be waiting days. Doing millions of them was therefore considered useless.
Even this is being gentle. A good case can be made that every component of these algorithms was known once differentiation was formalized, but nobody put them together, not because they didn't realize it could work, but because it was useless : doing things this way would have been incredibly inefficient compared to the then normal ways of doing things. E.g. finding formulas and constants by small adjustments on large chains of partial derivatives (ie. "Deep learning") is something that Isaac Newton knew how to do. It's just he would declare you totally mad for doing that. One might criticize that there were a few holes in the mathematical understanding of matrices in Newton's day, I wouldn't doubt that had he had a reason to really look into those problems, he would have fixed them. But he was more than 50 orders of magnitudes of : doing a single 50x50 matrix multiplication with any amount of resources then available wouldn't have finished before he died.
Now machine learning tutorials tell you to run unrolled 30x30 LSTM expansions on audio data and unroll for 50 datapoints. And then doing that millions of times (tens of thousands of times on a few thousand samples). You can expect this to run on the computer you're reading this on in a matter of hours.
- danieldk 11y agoJust like today's AI algorithms were known in the 1960's. But they were not. To take recurrent networks as an example, active work on RNNs only started in the 80ies. LSTMs, which are one of the solutions to the vanishing gradient problem were only discovered in 1997 by Sepp Hochreiter. Of course, computational power is one of the limiting factors, but saying that it is the only thing that held AI back is disingenuous and dismisses the hard work of a couple of generations in AI research. Edit: and Schmidhuber
- YeGoblynQueenne 11y ago>> LSTMs ... were only discovered in 1997 by Sepp Hochreiter. Aye. Hochreiter and the One Who Shall Not be Named. Let's not forget him again, eh?
- GFK_of_xmaspast 11y ago> Just like today's AI algorithms were known in the 1960's Does this mean anything more substantial than 'they knew about neural networks back then'.
- vonmoltke 11y ago> Just like today's AI algorithms were known in the 1960's > Now machine learning tutorials tell you to run unrolled 30x30 LSTM expansions Like the LSTM algorithm that wasn't published until 1997? > So why didn't we have live speech transcription in the 1960s ? We all know the answer : processing power was 50 orders of magnitude less than we needed. You are seriously underestimating the challenges inherent in the physical sensors required to perform things like speech recognition. Having done some AI/ML on surveillance radar I can tell you that getting reliable data on the physical phenomena is a big part of the challenge in teaching machines to learn and recognize what is going on.