4 ms·
Not sure what you mean by above, but happy to answer. First, I think you're referring most proximately to Marcus' paragraph including: > And it’s where we sho
by evrydayhustling 8y ago
Not sure what you mean by above, but happy to answer. First, I think you're referring most proximately to Marcus' paragraph including:
> And it’s where we should all be looking: gradient descent plus symbols, not gradient descent alone.
That's the most specifically defined proposal that I see in the article. It is a reasonable mission statement to fire up your lab or community about. It's also not something outside the ongoing deep learning discourse, and it's a direction I personally am excited about. There has been great work recently by DeepMind [1], for example, about using gradient descent to theorize about symbolic relationships.
However, the statement above is also nowhere near operationalized enough to say "agree / disagree" in any scientific sense. A specific model demonstrating advantages of the marrying symbolic and gradient-based reasoning (c.f. the one above) would open itself to productive discussion of its successes and failures. People have asked for this (including in the tweetstorm), but most weirdly Marcus seems to be responding that operationalizing a model or even a success criterion is itself a waste of time! Quoting:
> I actually think benchmarks are to some degree the wrong way forward, and have said that for two decades. People need to take a step back, and reflect on where things stand, rather than rushing into the next bakeoff.
I couldn't disagree with this statement more, and it brings to my biggest personal axe to grind about the "is this how we get to AGI" query: AGI itself hasn't been defined in a way that is compatible with empirical discourse. Machine Learning at a term in many ways exists to abandon "AI"'s association with AGI, because the community realized that focusing on success at specific, objectively measurable tasks would move them forward more effectively.
When people focused on AGI say that ML in its current form won't get there, my answer is: "It's almost tautological that we're going to need new methods to get to something called AGI, but we can't even discuss progress on it until you give a measurable definition".
I don't blame folks (including Marcus) for wanting to continue discussing AGI as an abstraction - maybe they will be the ones to find a good operational definition! But it's weird and unscientific to say people shouldn't continue work on other directions in the mean time, or to say that a specific technique is essential before either the technique or the end goal has been effectively defined.
[1] https://deepmind.com/blog/neural-approach-relational-reasoning/ https://deepmind.com/blog/neural-approach-relational-reasoni...
- woodandsteel 8y agoThank you for your response.