4 ms·
It's interesting to note here that the early years of machine learning (like the first couple of decades of it) were almost exclusively dedicated to learning sy
by YeGoblynQueenne 8y ago
It's interesting to note here that the early years of machine learning (like
the first couple of decades of it) were almost exclusively dedicated to
learning symbolic rules, primarily in propositional logic. So if you pick up
older machine learning papers, from around the '70s and '80s, you can see that
the "models" are almost always propositional logic rules [1].
To cut a very long story short, machine learning took off in part as an
attempt to automate knowledge acquisition for expert systems, the dominant AI
paradigm at the time. Somehow, for reasons that are not entirely clear to me,
the goal of learning the rules for a rule-based system was abandoned and since
then research has focused almost exlusively on just learning.
As a result of this, most machine learning work today does not consider what
one can do with a trained model. And yet, just having, say, an object
classifier, is not very useful on its own. For example, a robot car must be
able to take decisions based on the objects its machine vision algorithms
identify. Although there are learning techniques that consider this
particular problem, i.e. navigation, there is little work on general inference
and reasoning over the output of trained models.
And this is not a symbolic vs statistical thing. I work in symbolic machine
learning, and we are pretty rubbish at that, too. It's like we have lots of
little pieces of a puzzle all lying around and nobody is really trying to put
them together. Instead, we each carve our little (or bibger) niche and pretend
nothing exists outside of it.
Why? It's a mystery to me. Gary Marcus is doing good to try and shake up the
field a bit. We need to move on from our successes just as we move on from our
failures. And he's damn right about the how, too: symbolic AI is due for a
comeback.
____________________
[1] For instance, you can see that in the foundational theoretical text for
modern machine learning, Leslie Valiant's A Theory of the Learnable where
the learning framework is described in terms of the propositional calculus,
with a concept represented as a predicate that recognises vectors of boolean
variables as members of itself or not.
And of course, probably the most famous representative of propositional logic
learning is the class of algorithms we probably all know as decision trees;
they learn disjunctions of conjunctions - propositional logic rules.
- YeGoblynQueenne 8y ago>> The secondary goal of the book was to show that that was possible to build the primitives of symbol manipulation in principle using neurons as elements. Just another note here to say that the original artificial neuron, the Pitts and McCulloch neuron, was designed as a logic circuit with the ability to learn its own function. At the time -the 1930's- the propositional calculus was considered to be a good model of human intelligence.
- eli_gottlieb 8y ago>Why? It's a mystery to me. Gary Marcus is doing good to try and shake up the field a bit. We need to move on from our successes just as we move on from our failures. And he's damn right about the how, too: symbolic AI is due for a comeback. How would we define "symbolic", though? I'm about to give a presentation on some related matters next week, and I've been trying to think about how to avoid just saying, as Marcus is perceived to do, "lol psychology critiques AI so AI needs to use symbols." For example, there are loads and loads of symbolic AI techniques which don't demonstrate basic features of "symbolic" human cognition such as causal inference and productivity/compositionality, on top of symbolic techniques having essentially no way to address the Frame Problem.