3 ms·

Not everything that is AI is machine learning. The program in the article may be using decision trees but the program is not a learning algorithm but an expert

Not everything that is AI is machine learning. The program in the article may be using decision trees but the program is not a learning algorithm but an expert system. The difference is that machine learning is inductive, while expert systems are deductive [1]. Here's another example of doing the same thing in Prolog from Markus Triska's website, that also identifies animals by asking the user. Title "Expert Systems in Prolog": https://www.metalevel.at/prolog/expertsystems https://www.metalevel.at/prolog/expertsystems And here's another one that identifies birds, on the Amzi Prolog website: https://amzi.com/ExpertSystemsInProlog/02usingprolog.php https://amzi.com/ExpertSystemsInProlog/02usingprolog.php _____________ [1] Reasoning. Induction and deduction are forms of reasoning. You know, the thing all the LLM folks don't know how to define? We've been able to do that since before the '80s and decision trees are just one way to do it. Learning a decision tree, e.g. with a decision tree learner like ID3 or C4.5 is an inductive task. Using a (learned or hand-coded) decision tree to make a decision is a deductive one. But hey, we don't have a definition of reasoning. Oooh nooo.

YeGoblynQueenne 2y ago
There is a very important reason why "decision tree" is both a deductive inference procedure and the "model" (really, a propositional logic theory) learned by decision tree learners. The reason is that the way decision trees, well, make decisions, is the way that expert systems of the '80s made decisions. The first expert system was called DENDRAL (capitals mandatory), derived from the Greek word for "tree", "dendron" (also see: "dendrite"; little tree). DENDRAL and similar systems implement deductive inference procedures that traverse a tree (either bottom-up or top-down, with backward or forward chaining). In traditional expert systems the edges in the tree are production rules of the form IF-THEN-ELSE and the nodes are intermediary goals of what is essentially a proof of a propositional logic formula (the decision tree; it's a boolean formula). Expert systems' Achilles heel was the so-called "knowledge acquisition bottleneck" which refers, in short, to the difficulty of constructing and maintaining a huge rule-base as needed for an expert system that can do something more interesting than play 20 questions like the animal identification system above. Because of this difficulty, expert systems never quite fulfilled their promises and they suffered from the brittleness that is one of the first thing anyone hears about expert systems today. And what did people come up with in order to overcome the knowledge acquisition bottleneck? Why, machine learning, of course. The early days of machine learning as a field --remember, we're back in the '80s, a bit before the AI winter hits-- were exactly systems that tried to learn the IF-THEN-ELSE rules for expert systems' rule bases. Decision tree learners, like ID3 and C4.5 by J. Ross Quinlan, and similar algorithms, that were wildly popular even well after the ImageNet moment of deep neural nets come from that era. Instead of learning only the rules though, decision tree learners, learn an entire decision tree; in short, an entire expert system, from top to bottom.

YeGoblynQueenne 2y ago

There is a very important reason why "decision tree" is both a deductive inference procedure and the "model" (really, a propositional logic theory) learned by decision tree learners. The reason is that the way decision trees, well, make decisions, is the way that expert systems of the '80s made decisions. The first expert system was called DENDRAL (capitals mandatory), derived from the Greek word for "tree", "dendron" (also see: "dendrite"; little tree). DENDRAL and similar systems implement deductive inference procedures that traverse a tree (either bottom-up or top-down, with backward or forward chaining). In traditional expert systems the edges in the tree are production rules of the form IF-THEN-ELSE and the nodes are intermediary goals of what is essentially a proof of a propositional logic formula (the decision tree; it's a boolean formula). Expert systems' Achilles heel was the so-called "knowledge acquisition bottleneck" which refers, in short, to the difficulty of constructing and maintaining a huge rule-base as needed for an expert system that can do something more interesting than play 20 questions like the animal identification system above. Because of this difficulty, expert systems never quite fulfilled their promises and they suffered from the brittleness that is one of the first thing anyone hears about expert systems today. And what did people come up with in order to overcome the knowledge acquisition bottleneck? Why, machine learning, of course. The early days of machine learning as a field --remember, we're back in the '80s, a bit before the AI winter hits-- were exactly systems that tried to learn the IF-THEN-ELSE rules for expert systems' rule bases. Decision tree learners, like ID3 and C4.5 by J. Ross Quinlan, and similar algorithms, that were wildly popular even well after the ImageNet moment of deep neural nets come from that era. Instead of learning only the rules though, decision tree learners, learn an entire decision tree; in short, an entire expert system, from top to bottom.