3 ms·

Here we are in 1989 again. The cycle keeps repeating. A new advancement in computing power, networking, or algorithms means there's a new batch of low-hanging

Here we are in 1989 again. The cycle keeps repeating. A new advancement in computing power, networking, or algorithms means there's a new batch of low-hanging fruit for AI to pick, so we pick it. Investors say "What about the high-hanging fruit?" and we say "No problem. We just need a slightly longer ladder." Two years later everybody finally realizes the high-hanging fruit is on the moon.

gkolli 6y ago
thank you for this amazing analogy!
tgv 6y ago
My first AI teacher (even before '89) compared solving AI with neural nets to teaching pigs to fly by throwing them from a tower. Improvements come from building higher towers. There's a recent NLP model that was trained on a trillion words. It would take us 10,000 years to read or listen (no breaks, no sleep) to that many words. Problems like attention, and the relation between memory and sequential thinking haven't been cleared up at all. Even semantics, i.e. basic understanding of a utterance or a scene, is in its infancy. Large neural nets can help with interesting problems, but it's not going to mimic our style of thinking any time soon.
- mjburgess 6y ago
  It isn't ever going to model our style of thinking. A "neural network" is just high-dimension linear regression; the idea it has anything to do with the brain is metaphorical nonesense. No algorithm running on digital hardware can emulate the biological process of animal intelligence.
  - machiaweliczny 6y ago
    Why not?
    - mjburgess 6y ago
      What algorithm running on a digital computer would make the computer transparent?
snarf21 6y ago
It seems clear that were limited by access to data and computer power. That has largely changed. However, the hard part has always been around deciding what question to ask and how to score the result. This requires domain knowledge and a scientific process (conjecture, measure, analyze, repeat). There is no magic bullet just better tools. Someone still has to define the problem and solution. Just look at IBM Watson and what a failure that is. They did okay on Jeopardy because they spent years optimizing for just that. Jeopardy could have changed the way they wrote clues by including irony and other language constructs and the humans would have done fine and Watson would have bombed.

gkolli 6y ago

thank you for this amazing analogy!

tgv 6y ago

My first AI teacher (even before '89) compared solving AI with neural nets to teaching pigs to fly by throwing them from a tower. Improvements come from building higher towers. There's a recent NLP model that was trained on a trillion words. It would take us 10,000 years to read or listen (no breaks, no sleep) to that many words. Problems like attention, and the relation between memory and sequential thinking haven't been cleared up at all. Even semantics, i.e. basic understanding of a utterance or a scene, is in its infancy. Large neural nets can help with interesting problems, but it's not going to mimic our style of thinking any time soon.

mjburgess 6y ago
It isn't ever going to model our style of thinking. A "neural network" is just high-dimension linear regression; the idea it has anything to do with the brain is metaphorical nonesense. No algorithm running on digital hardware can emulate the biological process of animal intelligence.
- machiaweliczny 6y ago
  Why not?
  - mjburgess 6y ago
    What algorithm running on a digital computer would make the computer transparent?

snarf21 6y ago

It seems clear that were limited by access to data and computer power. That has largely changed. However, the hard part has always been around deciding what question to ask and how to score the result. This requires domain knowledge and a scientific process (conjecture, measure, analyze, repeat). There is no magic bullet just better tools. Someone still has to define the problem and solution. Just look at IBM Watson and what a failure that is. They did okay on Jeopardy because they spent years optimizing for just that. Jeopardy could have changed the way they wrote clues by including irony and other language constructs and the humans would have done fine and Watson would have bombed.