4 ms·

No, they don't. Now, feel free to _show_ me that they do, and I will gladly accept to have been wrong. But this argument that because of some very theoretical

by 317070 8y ago

No, they don't. Now, feel free to _show_ me that they do, and I will gladly accept to have been wrong. But this argument that because of some very theoretical view on the problem the current engineering solutions should be abandoned, without actually providing good engineering alternatives, is weird. I have a bit of a "not even wrong" feeling on the symbolist side of the debate. Deep neural networks have serious flaws, but they can at least be pointed out due to them already solving engineering problems. It is easy to say a field should move to your direction, but they will probably only do it when provided with tangible evidence. AI has moved into the realm of application, and that means the scientific bar for purely theoretical ideas has been raised compared to 20 years ago. And right now, the tangible evidence for symbolism is simply not cutting it.

scotty79 8y ago
I think they need symbols but they just need to be taught symbols. If you kept child stationary and show it just static photos one at a time you wouldn't be surprised that it can't tell bus lying on the side in the snow from a snow plow. What neural networks need is the same thing little children need, motion, rotation, things stacking hitting and occluding each other ... and possibly even interaction. We haven't taught NN to see properly yet and guy wants to hardwire math into them because he thinks that's the only option. Humans learn symbols late and only with deliberate effort.
- Asooka 8y ago
  On the other hand humans have built-in special processing stages already. One that comes to mind is that the neurons in your eyes in effect run a sharpening filter on the incoming light before the data ever enters your brain. I don't think it's a stretch to propose that we should implement certain features as special-purpose processing stages that the neural network can use, with the goal of having a more efficient system. The holy grail would be a neural network that has programming capabilities and can write bare metal programs for itself.
  - scotty79 8y ago
    > On the other hand humans have built-in special processing stages already. Yes. But there were cases where human could perform function elsewhere that was usually done in special site when thst site got damaged.
stult 8y ago
Or to frame your argument in terms of science rather than engineering, it's a falsifiable hypothesis. The author should go try to test it and then let us know if it works.
- bsaul 8y ago
  i’d say the bus picture classification is a pretty telling example that something’s missing in the amount of knowledge deep learning is able to extract from training data sets. Anybody who understand what a school bus is wouldn’t mistake it for a snow plow in the other pictures.
vanderZwan 8y ago
> And right now, the tangible evidence for symbolism is simply not cutting it. In AI/ML perhaps. In the biological and psychological sciences the importance of symbols to cognitive functioning is pretty well established. Just look at how important the mastery of language is for complicated thought. One of the main differences between linguistic thought and other forms of thinking is all about symbol manipulation. (This says nothing about the quality of this article of course)
- FakeComments 8y ago
  It’s also pretty well known in math and CS/SE communities. Heck, the mathematical research into the power of language, and in particular the ability to embed models of reasoning into equations to allow you to formalize your meta-reasoning, is what led to the birth of electric computers. So I find it shallow skepticism to suggest there’s not evidence that it would be useful to AI, because literally the most cursory glance would show mounds of it. The real challenge is and always has been how to combine or embed symbolic reasoning capabilities efficiently within a fuzzier approximator, allowing for “getting close” in a fuzzy way then fine tuning it using more advanced rules. That mimics more closely how we think, where we broadly intuit a few potential answers, then use higher order thinking to sift through them.
  - vanderZwan 8y ago
    > The real challenge is and always has been how to combine or embed symbolic reasoning capabilities efficiently within a fuzzier approximator, allowing for “getting close” in a fuzzy way then fine tuning it using more advanced rules. Well, maybe the problem is that we spent so much time basing models on the visual part of the brain that we overlooked the rest: https://www.quantamagazine.org/new-ai-strategy-mimics-how-brains-learn-to-smell-20180918/ https://www.quantamagazine.org/new-ai-strategy-mimics-how-br...
chimprich 8y ago
> But this argument that because of some very theoretical view on the problem the current engineering solutions should be abandoned That's not the author's argument at all. Even the subtitle of the article says "We’ll need both deep learning and symbol manipulation to build AI.".
- posterboy 8y ago
  A compromise between the two would be to identify powerful abstructions over nodes and nets that are equivalent to symbol manipulation circuits. Self assembly and self modification are certainly not strange to symbolic logic, but this doesn't work so well for most programmers. So we need more geniuses and they need to tailor neural nets to help them build symbolic logic (if only as intermediate debugging output).
drb91 8y ago
Getting tasks done has never been a ringing endorsement for the general applicability of the underlying algorithm. Meanwhile, the entirety of human communication is built on symbols. If computers can’t deal with them, how useful can computers ultimately be? Roombas can only improve so much.
giardini 8y ago
If you're willing to accept a (possible) inexplicable intelligence that cannot explain its decisions or reasoning processes then you'd be fine w/o symbolism. In fact there may be an example for you (perhaps in your pocket): The next time you have a decision to make, phrase it as a Heads/Tails question and flip a coin. Over its time of use, the coin will be right until it is wrong, putting you in exactly the same situation as you would be with an inexplicable intelligence. So you see, intelligence is not that far away! But explicable predictive intelligence requires reasoned prediction and can answer "Why that prediction?" when asked and can learn if a prediction fails. The coin cannot but will be adequate, at least until it fails sufficiently to induce you to abandon it.

by 317070 8y ago

scotty79 8y ago

I think they need symbols but they just need to be taught symbols. If you kept child stationary and show it just static photos one at a time you wouldn't be surprised that it can't tell bus lying on the side in the snow from a snow plow. What neural networks need is the same thing little children need, motion, rotation, things stacking hitting and occluding each other ... and possibly even interaction. We haven't taught NN to see properly yet and guy wants to hardwire math into them because he thinks that's the only option. Humans learn symbols late and only with deliberate effort.

Asooka 8y ago
On the other hand humans have built-in special processing stages already. One that comes to mind is that the neurons in your eyes in effect run a sharpening filter on the incoming light before the data ever enters your brain. I don't think it's a stretch to propose that we should implement certain features as special-purpose processing stages that the neural network can use, with the goal of having a more efficient system. The holy grail would be a neural network that has programming capabilities and can write bare metal programs for itself.
- scotty79 8y ago
  > On the other hand humans have built-in special processing stages already. Yes. But there were cases where human could perform function elsewhere that was usually done in special site when thst site got damaged.

stult 8y ago

Or to frame your argument in terms of science rather than engineering, it's a falsifiable hypothesis. The author should go try to test it and then let us know if it works.

bsaul 8y ago
i’d say the bus picture classification is a pretty telling example that something’s missing in the amount of knowledge deep learning is able to extract from training data sets. Anybody who understand what a school bus is wouldn’t mistake it for a snow plow in the other pictures.

vanderZwan 8y ago

> And right now, the tangible evidence for symbolism is simply not cutting it. In AI/ML perhaps. In the biological and psychological sciences the importance of symbols to cognitive functioning is pretty well established. Just look at how important the mastery of language is for complicated thought. One of the main differences between linguistic thought and other forms of thinking is all about symbol manipulation. (This says nothing about the quality of this article of course)

FakeComments 8y ago
It’s also pretty well known in math and CS/SE communities. Heck, the mathematical research into the power of language, and in particular the ability to embed models of reasoning into equations to allow you to formalize your meta-reasoning, is what led to the birth of electric computers. So I find it shallow skepticism to suggest there’s not evidence that it would be useful to AI, because literally the most cursory glance would show mounds of it. The real challenge is and always has been how to combine or embed symbolic reasoning capabilities efficiently within a fuzzier approximator, allowing for “getting close” in a fuzzy way then fine tuning it using more advanced rules. That mimics more closely how we think, where we broadly intuit a few potential answers, then use higher order thinking to sift through them.
- vanderZwan 8y ago
  > The real challenge is and always has been how to combine or embed symbolic reasoning capabilities efficiently within a fuzzier approximator, allowing for “getting close” in a fuzzy way then fine tuning it using more advanced rules. Well, maybe the problem is that we spent so much time basing models on the visual part of the brain that we overlooked the rest: https://www.quantamagazine.org/new-ai-strategy-mimics-how-brains-learn-to-smell-20180918/ https://www.quantamagazine.org/new-ai-strategy-mimics-how-br...

chimprich 8y ago

> But this argument that because of some very theoretical view on the problem the current engineering solutions should be abandoned That's not the author's argument at all. Even the subtitle of the article says "We’ll need both deep learning and symbol manipulation to build AI.".

posterboy 8y ago
A compromise between the two would be to identify powerful abstructions over nodes and nets that are equivalent to symbol manipulation circuits. Self assembly and self modification are certainly not strange to symbolic logic, but this doesn't work so well for most programmers. So we need more geniuses and they need to tailor neural nets to help them build symbolic logic (if only as intermediate debugging output).

drb91 8y ago

Getting tasks done has never been a ringing endorsement for the general applicability of the underlying algorithm. Meanwhile, the entirety of human communication is built on symbols. If computers can’t deal with them, how useful can computers ultimately be? Roombas can only improve so much.

giardini 8y ago

If you're willing to accept a (possible) inexplicable intelligence that cannot explain its decisions or reasoning processes then you'd be fine w/o symbolism. In fact there may be an example for you (perhaps in your pocket): The next time you have a decision to make, phrase it as a Heads/Tails question and flip a coin. Over its time of use, the coin will be right until it is wrong, putting you in exactly the same situation as you would be with an inexplicable intelligence. So you see, intelligence is not that far away! But explicable predictive intelligence requires reasoned prediction and can answer "Why that prediction?" when asked and can learn if a prediction fails. The coin cannot but will be adequate, at least until it fails sufficiently to induce you to abandon it.