4 ms·

I didn't say they didn't! Yes, they "bring in" probabilistic programming, graphical representations of symbolic structures, and more. But all of the citations

I didn't say they didn't! Yes, they "bring in" probabilistic programming, graphical representations of symbolic structures, and more. But all of the citations 6-10 are from papers that integrate new approaches into a deep neural model. Most are from self- described deep learning teams, and all would be welcome at conferences and publications within the deep learning community. Integrating with other inference paradigms has been a major driver in neural network research for 30y... so trying to cast progress like this as some sort of comeuppance that proves the insufficiency of the field is BS.

eli_gottlieb 8y ago
>But all of the citations 6-10 are from papers that integrate new approaches into a deep neural model. Having read citation 8, "Building Machines that Learn and Think like People", a number of times over, I don't see where that paper includes a neural network architecture.
- sh33mp 8y ago
  The above commenter probably spoke a little too hastily. "Building Machines" is indeed not a paper about a neural network method, but a survey of problems that we should expect neural networks to do better on, but they currently aren't. That said, the paper isn't down on deep learning, but rather says "we need extra stuff, either additional methods or more inductive biases in our models".
- evrydayhustling 8y ago
  Your correct - that one doesn't propose an architecture but suggests that others do so. It also is arguably from "outside" the main deep learning community, and in the context of the OP could be interpreted as corrective. However, that is exactly not the paper's own stance, which is explicitly constructive to the field. And it is reciprocated! Check out all the NeurIPS papers citing and relating their work to the suggestions here: https://scholar.google.com/scholar?cites=8504723689348856287&as_sdt=5,33&sciodt=0,33&hl=en https://scholar.google.com/scholar?cites=8504723689348856287... While I misspoke, my point remains that both this contribution and it's reception are part of a healthy community, not a case of "moving past" or exposing shortcomings of a field..
  - eli_gottlieb 8y ago
    >While I misspoke, my point remains that both this contribution and it's reception are part of a healthy community, not a case of "moving past" or exposing shortcomings of a field.. From within "the field", there isn't a single "deep learning community" counterposed to an "other stuff community". There's a broad machine learning community, in which deep learning is currently the most fashionable approach, and there are small factions of pro- and anti-"deep" partisans, who consider their approach the One True Approach. Science is usually unkind to partisans, and so most people I've met don't really bother with that kind of partisanship. Asking, "Can deep learning get us all the way to AGI, yes or no?" is the wrong question. The proper questions are: * What is deep learning good or bad at? * What do we mean by AGI? (Similarly: data analytics, machine perception, statistical AI, etc?) * Why should we target one goal (high-dimensional statistics, missing-data reconstruction, reinforcement learning, AGI) over another? * What methods currently afford us the best contributions towards the scientific and engineering goals we're targeting?
    - evrydayhustling 8y ago
      Sounds like we are on exactly the same page. Defining "the field" precisely is a rhetorical device. Defining and working on problems is what creates progress.

eli_gottlieb 8y ago

>But all of the citations 6-10 are from papers that integrate new approaches into a deep neural model. Having read citation 8, "Building Machines that Learn and Think like People", a number of times over, I don't see where that paper includes a neural network architecture.

sh33mp 8y ago
The above commenter probably spoke a little too hastily. "Building Machines" is indeed not a paper about a neural network method, but a survey of problems that we should expect neural networks to do better on, but they currently aren't. That said, the paper isn't down on deep learning, but rather says "we need extra stuff, either additional methods or more inductive biases in our models".
evrydayhustling 8y ago
Your correct - that one doesn't propose an architecture but suggests that others do so. It also is arguably from "outside" the main deep learning community, and in the context of the OP could be interpreted as corrective. However, that is exactly not the paper's own stance, which is explicitly constructive to the field. And it is reciprocated! Check out all the NeurIPS papers citing and relating their work to the suggestions here: https://scholar.google.com/scholar?cites=8504723689348856287&as_sdt=5,33&sciodt=0,33&hl=en https://scholar.google.com/scholar?cites=8504723689348856287... While I misspoke, my point remains that both this contribution and it's reception are part of a healthy community, not a case of "moving past" or exposing shortcomings of a field..
- eli_gottlieb 8y ago
  >While I misspoke, my point remains that both this contribution and it's reception are part of a healthy community, not a case of "moving past" or exposing shortcomings of a field.. From within "the field", there isn't a single "deep learning community" counterposed to an "other stuff community". There's a broad machine learning community, in which deep learning is currently the most fashionable approach, and there are small factions of pro- and anti-"deep" partisans, who consider their approach the One True Approach. Science is usually unkind to partisans, and so most people I've met don't really bother with that kind of partisanship. Asking, "Can deep learning get us all the way to AGI, yes or no?" is the wrong question. The proper questions are: * What is deep learning good or bad at? * What do we mean by AGI? (Similarly: data analytics, machine perception, statistical AI, etc?) * Why should we target one goal (high-dimensional statistics, missing-data reconstruction, reinforcement learning, AGI) over another? * What methods currently afford us the best contributions towards the scientific and engineering goals we're targeting?
  - evrydayhustling 8y ago
    Sounds like we are on exactly the same page. Defining "the field" precisely is a rhetorical device. Defining and working on problems is what creates progress.