6 ms·

I don’t like how they oppose deep learning with « logic based AI ». When Hinton got great results, he wasn’t competing against logic based AI but rather other m

I don’t like how they oppose deep learning with « logic based AI ». When Hinton got great results, he wasn’t competing against logic based AI but rather other machine learning approaches like SVMs.

etangent 7y ago
Sure, but for someone who does not know what an SVM is, the big-picture takeaway is that a particular version of "soft" statistical learning (deep learning) had succeeded where "hard logic" approach failed.
- justicezyx 7y ago
  Just asking: hard logic failed in what area?
  - iezepov 7y ago
    Computer Vision would the the most likely example. It’s quite hard to come up with “hard rules” for distinguishing cats and dogs.
  - tomcam 7y ago
    Speech recognition is one such case. We thought that by breaking down the incoming speech into phonemes we would be able to get words from the bottom up. That never worked too well, whereas statistical speech recognition has been much more effective.
  - conjectures 7y ago
    Carrots are vegetables. Vegetables are edible. Is this a picture of something edible? Having a bunch of propositions in a database is not much help if you can't relate them to messy sensory input. It turns out that making sense out of mess was a hard problem. Eventually, logic* will have to be brought in again, but at a higher level, to help out. * Ofc Hinton's approach is based on logic too, but in the form of probability, calculus etc.
    - DoctorOetker 7y ago
      but with probe metrics and other experiments we are retroactively explaining why the big breaktroughs actually work, and these can inspire and fix the big breakthrough by putting them on more rigorous foundations. At the end you get a formal explanation of the conditional probability that given a carrot with 6DoF and such and such parameters (age, size, ...) in such and such lighting conditions results in that probability distribution for pixel values.
nextos 7y ago
Me neither. In fact, both will be important components of future AI. I like to see deep learning as intuition, and logic-based AI as formal reasoning. Intuition allows systems to avoid search problems, to some extent. Formal reasoning introduces structure.
buboard 7y ago
I dont think he was competing. ANNs and boltzmann machines in the 80s seemed was an exploratory field not really part of statistical ML. At the time people were indeed competing with logic-based AI. SVMs came later and even then i believe, NNs had a different history and were not motivated from competition to SOTA techniques like SVMs. When the success of trainig came after the mid-2000s that's when ANNs started being called "ML". At least that's my memory
- kiterunner2346 7y ago
  Indeed, he wasn't competing. In fact, he wasn't moving; he kept doing what he had always been doing until, luckily, something, that he had nothing to do with, changed and allowed NN methods to work better. So I view him as essentially standing still, frozen in time, like a clock stuck at 6 o'clock. And, just like that clock, he would inevitably be correct (at least twice a day). He had nothing to do with the tools (higher memory & CPU speeds) that made his methods work, he just kept doing the same thing over and over until, one day, by accident, something important changed: his lab bought newer, faster computers. He's not exactly Louis Pasteur is what I'm saying!
  - buboard 7y ago
    he s still the coinventor of boltzmann machines, and one who kept the field alive.
  - jbay808 7y ago
    We call this "skating to where the puck is going to be".
    - kiterunner2346 7y ago
      I call it "doing the same thing over and over again, hoping someday you get lucky." I think there's a commonly-used expression for that behavior. Hinton got lucky: something he had nothing to do with changed, making him look like a stoic hero. A winning strategy or just a lazy path?
Upvoter33 7y ago
Well, Hinton himself makes a big deal of this in his Turing award speech. See: https://fcrc.acm.org/turing-lecture-at-fcrc-2019 https://fcrc.acm.org/turing-lecture-at-fcrc-2019

etangent 7y ago

Sure, but for someone who does not know what an SVM is, the big-picture takeaway is that a particular version of "soft" statistical learning (deep learning) had succeeded where "hard logic" approach failed.

justicezyx 7y ago
Just asking: hard logic failed in what area?
- iezepov 7y ago
  Computer Vision would the the most likely example. It’s quite hard to come up with “hard rules” for distinguishing cats and dogs.
- tomcam 7y ago
  Speech recognition is one such case. We thought that by breaking down the incoming speech into phonemes we would be able to get words from the bottom up. That never worked too well, whereas statistical speech recognition has been much more effective.
- conjectures 7y ago
  Carrots are vegetables. Vegetables are edible. Is this a picture of something edible? Having a bunch of propositions in a database is not much help if you can't relate them to messy sensory input. It turns out that making sense out of mess was a hard problem. Eventually, logic* will have to be brought in again, but at a higher level, to help out. * Ofc Hinton's approach is based on logic too, but in the form of probability, calculus etc.
  - DoctorOetker 7y ago
    but with probe metrics and other experiments we are retroactively explaining why the big breaktroughs actually work, and these can inspire and fix the big breakthrough by putting them on more rigorous foundations. At the end you get a formal explanation of the conditional probability that given a carrot with 6DoF and such and such parameters (age, size, ...) in such and such lighting conditions results in that probability distribution for pixel values.

nextos 7y ago

Me neither. In fact, both will be important components of future AI. I like to see deep learning as intuition, and logic-based AI as formal reasoning. Intuition allows systems to avoid search problems, to some extent. Formal reasoning introduces structure.

buboard 7y ago

I dont think he was competing. ANNs and boltzmann machines in the 80s seemed was an exploratory field not really part of statistical ML. At the time people were indeed competing with logic-based AI. SVMs came later and even then i believe, NNs had a different history and were not motivated from competition to SOTA techniques like SVMs. When the success of trainig came after the mid-2000s that's when ANNs started being called "ML". At least that's my memory

kiterunner2346 7y ago
Indeed, he wasn't competing. In fact, he wasn't moving; he kept doing what he had always been doing until, luckily, something, that he had nothing to do with, changed and allowed NN methods to work better. So I view him as essentially standing still, frozen in time, like a clock stuck at 6 o'clock. And, just like that clock, he would inevitably be correct (at least twice a day). He had nothing to do with the tools (higher memory & CPU speeds) that made his methods work, he just kept doing the same thing over and over until, one day, by accident, something important changed: his lab bought newer, faster computers. He's not exactly Louis Pasteur is what I'm saying!
- buboard 7y ago
  he s still the coinventor of boltzmann machines, and one who kept the field alive.
- jbay808 7y ago
  We call this "skating to where the puck is going to be".
  - kiterunner2346 7y ago
    I call it "doing the same thing over and over again, hoping someday you get lucky." I think there's a commonly-used expression for that behavior. Hinton got lucky: something he had nothing to do with changed, making him look like a stoic hero. A winning strategy or just a lazy path?

Upvoter33 7y ago

Well, Hinton himself makes a big deal of this in his Turing award speech. See: https://fcrc.acm.org/turing-lecture-at-fcrc-2019 https://fcrc.acm.org/turing-lecture-at-fcrc-2019