3 ms·

I'm very disappointed with the quality of discussion between Marcus and LeCun. First Marcus responds to Bengio's interview with the arrogant and pretentious "I

by _cs2017_ 8y ago

I'm very disappointed with the quality of discussion between Marcus and LeCun. First Marcus responds to Bengio's interview with the arrogant and pretentious "I told you so" tweet, instead of simply saying that he agreed with Bengio. Then LeCun tweets a snarky and disrespectful response, instead of simply saying that Bengio's comments are different than Marcus's earlier critique. As a result of these tweets, I lost a huge amount of respect for both scientists.

thaw13579 8y ago
While I tend to agree with the authors view, I'm also disappointed by the tone. The author seems to double down on the "I told you so", saying: "I stand by that — which as far as I know (and I could be wrong) is the first place where anybody said that deep learning per se wouldn’t be a panacea, and would instead need to work in a larger context to solve a certain class of problems. Bengio was pretty much saying the same thing." which stakes out a priority claim (that deep learning is not a panacea) that appears somewhat superficial given the huge literature on causal modeling and shortcomings of the alternatives.
evrydayhustling 8y ago
While I agree that nobody looks great here, LeCun's response deserves to be read in context. Marcus' comment (and the linked post) isn't primarily about Bengio, it's about how he should be recognized as deep learning's earliest and most important public detractor. So, LeCun's response isn't about Marcus != Bengio, it's about the fact the Marcus' critique fundamentally hasn't deserved response or recognition this whole time, because there's not a constructive way to engage. That's a totally fair point to make, though a guy in LeCun's position probably should have said, "Gary, none of us have ever thought Deep Learning was (already) an answer for everything. The primary work of this community is to make it better."
- woodandsteel 8y ago
  To repeat my question above: Marcus is saying that to get AGI, or at least get a lot closer to it, we need to combine ML with symbolic manipulation. Do you agree or disagree?
  - evrydayhustling 8y ago
    Not sure what you mean by above, but happy to answer. First, I think you're referring most proximately to Marcus' paragraph including: > And it’s where we should all be looking: gradient descent plus symbols, not gradient descent alone. That's the most specifically defined proposal that I see in the article. It is a reasonable mission statement to fire up your lab or community about. It's also not something outside the ongoing deep learning discourse, and it's a direction I personally am excited about. There has been great work recently by DeepMind [1], for example, about using gradient descent to theorize about symbolic relationships. However, the statement above is also nowhere near operationalized enough to say "agree / disagree" in any scientific sense. A specific model demonstrating advantages of the marrying symbolic and gradient-based reasoning (c.f. the one above) would open itself to productive discussion of its successes and failures. People have asked for this (including in the tweetstorm), but most weirdly Marcus seems to be responding that operationalizing a model or even a success criterion is itself a waste of time! Quoting: > I actually think benchmarks are to some degree the wrong way forward, and have said that for two decades. People need to take a step back, and reflect on where things stand, rather than rushing into the next bakeoff. I couldn't disagree with this statement more, and it brings to my biggest personal axe to grind about the "is this how we get to AGI" query: AGI itself hasn't been defined in a way that is compatible with empirical discourse. Machine Learning at a term in many ways exists to abandon "AI"'s association with AGI, because the community realized that focusing on success at specific, objectively measurable tasks would move them forward more effectively. When people focused on AGI say that ML in its current form won't get there, my answer is: "It's almost tautological that we're going to need new methods to get to something called AGI, but we can't even discuss progress on it until you give a measurable definition". I don't blame folks (including Marcus) for wanting to continue discussing AGI as an abstraction - maybe they will be the ones to find a good operational definition! But it's weird and unscientific to say people shouldn't continue work on other directions in the mean time, or to say that a specific technique is essential before either the technique or the end goal has been effectively defined. [1] https://deepmind.com/blog/neural-approach-relational-reasoning/ https://deepmind.com/blog/neural-approach-relational-reasoni...
    - woodandsteel 8y ago
- _cs2017_ 8y ago
  Agree with everything you said. And I wish LeCun said exactly what you suggested. On second thought, I wish I myself was always very thoughtful and respectful in responding to questionable statements :)
- innagadadavida 8y ago
  It is LeCun’s personality and ego that’s responsible here. Other scientists are just explotiing it to get publicity and attention. For example see the Ali Rahimi’s alchemy speech in NIPS and the fallout: https://medium.com/@Synced/lecun-vs-rahimi-has-machine-learning-become-alchemy-21cb1557920d https://medium.com/@Synced/lecun-vs-rahimi-has-machine-learn...
- AndrewKemendo 8y ago
  LeCun basically said exactly that: Since we have DL but don't yet have AGI, it's quite obvious to everyone that we are missing something. I repeat: the interesting question is precisely what. I repeat my claim: DL (defined as gradient-based learning of non-linear functions) will be part of the solution. [1] https://twitter.com/ylecun/status/1067114490955735040 https://twitter.com/ylecun/status/1067114490955735040
amelius 8y ago
LeCun working for Facebook was already a disappointment.
Eridrus 8y ago
Marcus has been arguing against straw men without actually helping at all for decades now, it's entirely uninteresting to keep listening to him.

thaw13579 8y ago

While I tend to agree with the authors view, I'm also disappointed by the tone. The author seems to double down on the "I told you so", saying: "I stand by that — which as far as I know (and I could be wrong) is the first place where anybody said that deep learning per se wouldn’t be a panacea, and would instead need to work in a larger context to solve a certain class of problems. Bengio was pretty much saying the same thing." which stakes out a priority claim (that deep learning is not a panacea) that appears somewhat superficial given the huge literature on causal modeling and shortcomings of the alternatives.

evrydayhustling 8y ago

While I agree that nobody looks great here, LeCun's response deserves to be read in context. Marcus' comment (and the linked post) isn't primarily about Bengio, it's about how he should be recognized as deep learning's earliest and most important public detractor. So, LeCun's response isn't about Marcus != Bengio, it's about the fact the Marcus' critique fundamentally hasn't deserved response or recognition this whole time, because there's not a constructive way to engage. That's a totally fair point to make, though a guy in LeCun's position probably should have said, "Gary, none of us have ever thought Deep Learning was (already) an answer for everything. The primary work of this community is to make it better."

woodandsteel 8y ago
To repeat my question above: Marcus is saying that to get AGI, or at least get a lot closer to it, we need to combine ML with symbolic manipulation. Do you agree or disagree?
- evrydayhustling 8y ago
  Not sure what you mean by above, but happy to answer. First, I think you're referring most proximately to Marcus' paragraph including: > And it’s where we should all be looking: gradient descent plus symbols, not gradient descent alone. That's the most specifically defined proposal that I see in the article. It is a reasonable mission statement to fire up your lab or community about. It's also not something outside the ongoing deep learning discourse, and it's a direction I personally am excited about. There has been great work recently by DeepMind [1], for example, about using gradient descent to theorize about symbolic relationships. However, the statement above is also nowhere near operationalized enough to say "agree / disagree" in any scientific sense. A specific model demonstrating advantages of the marrying symbolic and gradient-based reasoning (c.f. the one above) would open itself to productive discussion of its successes and failures. People have asked for this (including in the tweetstorm), but most weirdly Marcus seems to be responding that operationalizing a model or even a success criterion is itself a waste of time! Quoting: > I actually think benchmarks are to some degree the wrong way forward, and have said that for two decades. People need to take a step back, and reflect on where things stand, rather than rushing into the next bakeoff. I couldn't disagree with this statement more, and it brings to my biggest personal axe to grind about the "is this how we get to AGI" query: AGI itself hasn't been defined in a way that is compatible with empirical discourse. Machine Learning at a term in many ways exists to abandon "AI"'s association with AGI, because the community realized that focusing on success at specific, objectively measurable tasks would move them forward more effectively. When people focused on AGI say that ML in its current form won't get there, my answer is: "It's almost tautological that we're going to need new methods to get to something called AGI, but we can't even discuss progress on it until you give a measurable definition". I don't blame folks (including Marcus) for wanting to continue discussing AGI as an abstraction - maybe they will be the ones to find a good operational definition! But it's weird and unscientific to say people shouldn't continue work on other directions in the mean time, or to say that a specific technique is essential before either the technique or the end goal has been effectively defined. [1] https://deepmind.com/blog/neural-approach-relational-reasoning/ https://deepmind.com/blog/neural-approach-relational-reasoni...
  - woodandsteel 8y ago
_cs2017_ 8y ago
Agree with everything you said. And I wish LeCun said exactly what you suggested. On second thought, I wish I myself was always very thoughtful and respectful in responding to questionable statements :)
innagadadavida 8y ago
It is LeCun’s personality and ego that’s responsible here. Other scientists are just explotiing it to get publicity and attention. For example see the Ali Rahimi’s alchemy speech in NIPS and the fallout: https://medium.com/@Synced/lecun-vs-rahimi-has-machine-learning-become-alchemy-21cb1557920d https://medium.com/@Synced/lecun-vs-rahimi-has-machine-learn...
AndrewKemendo 8y ago
LeCun basically said exactly that: Since we have DL but don't yet have AGI, it's quite obvious to everyone that we are missing something. I repeat: the interesting question is precisely what. I repeat my claim: DL (defined as gradient-based learning of non-linear functions) will be part of the solution. [1] https://twitter.com/ylecun/status/1067114490955735040 https://twitter.com/ylecun/status/1067114490955735040

amelius 8y ago

LeCun working for Facebook was already a disappointment.

Eridrus 8y ago

Marcus has been arguing against straw men without actually helping at all for decades now, it's entirely uninteresting to keep listening to him.