9 ms·

GOFAI didn't work. ML is the only AI approach that ever did. Sure, your tree searches and your planners and your grammars and your knowledge graphs and your exp

GOFAI didn't work. ML is the only AI approach that ever did. Sure, your tree searches and your planners and your grammars and your knowledge graphs and your expert systems still have their places... but they're not AI, and they never will be.

deleted 6y ago
[deleted]
YeGoblynQueenne 6y ago
Amicably, can I ask you how you know what you say above to be true? Where does your knowledge of AI come from? Edit: I'm asking because there is evident in your comment a confusion that is all too common on the internets today, of thinking of "GOFAI" (symbolic AI) and "ML" (machine learning) as two somehow incompatible and perhaps exclusive sub-fields of AI. This is as far from the truth as it could be, for instance machine learning really took off as a subject of research in the 1980's with the work of Ryszard Michalski and others, who considered machine learning approaches for the purpose of overcoming the "knowledge acquisition bottleneck", i.e. the difficulty of hand-crafting production rules for expert systems. Indeed, most of the early machine learning systems were propositional-logic based, i.e. symbolic. And of course, one of the most widely used and well-known type of machine learning approach used to this day, decision trees, hail from that time and also learn propositional logic, symbolic models. Of course, most people today know "ML" as a byword for deep learning, or at best statistical pattern recognition (if that). It's just another aberration brought on by the sudden explosion of interest in a once small field. I refer you to Michalski's textbook, "Machine Learning: An Artificial Intelligence Approach" and Tom Mitchell's "Machine Learning" for more information on the early days of the field, i.e. until ca. 2000.
- Veedrac 6y ago
  It doesn't much matter what people wrongly thought in the past, any more than one has to study the four fundamental elements of water, air, earth and fire to become an expert in chemistry. GOFAI didn't work, hence the winter. If it did I'd think otherwise. It's true that you can, in some places, merge the two. The significant majority of the time, this just makes your ML system worse in the long run, per The Bitter Lesson. Occasionally—very occasionally—your problem is fundamentally simple, so something brittle like AlphaZero works, even though we'd still rather shave the fixed-function parts off with MuZero and the like. But it's no coincidence this is reserved for simple, thoughtless problems (like brute-force move search) and kept isolated from the thinking, as intelligence needs generalization and abstraction, and GOFAI doesn't generalize or abstract. > And of course, one of the most widely used and well-known type of machine learning approach used to this day, decision trees Though decision trees live a healthy life in data analysis, alongside things like k-means clustering, they're obviously not AI. > Of course, most people today know "ML" as a byword for deep learning, or at best statistical pattern recognition (if that). ‘Statistical pattern recognition’ is just name calling devoid of real criticisms. I can ask GPT-3, > a = ["fitness", "health", "heart"], b = ["lifting", "curls", "squats"], c = ["running", "jogging"], so what is b.append("pushups")? and it'll happily answer > b.append("pushups") returns ["lifting", "curls", "squats", "pushups"] or I can give it the Loebner Prize questions which it can almost ace, or I can test it on 10-digit addition and it gets 60% right (with thousands separators, but still suffering BPEs), or I can ask it to do quirky tasks like shuffling letters, or ask it to write sentences with novel words and find that it understands nuanced differences between the definitions... You can scream ‘but it's just statistical pattern recognition’ all you want, but the criticism doesn't mean anything if ‘statistical pattern recognition’ includes this level of generalization, reasoning and algorithmic sophistication in natural language with scaling curves that assure us the best is yet to come.
  - YeGoblynQueenne 6y ago
    I "can scream all I want"? Before I continue this conversation, I'd like to ask, do you think this is an appropriate response to my comment? I responded to your original comment respectfully and politely. Note that I did not use "pattern recognition" in a derogatory manner, neither did I say that anything is "just" pattern recognition. I think you may be mistaking my comment for a different opinion that you disagree with.
    - YeGoblynQueenne 6y ago
      While waiting for Veedrac's reply to my comment, I thought I'd clarify here what I meant by "statistical pattern recognition". "Pattern recognition" is the name of the sub-field of AI research from which statistical machine learning grew into what it is today, for example many problems in machine vision are typically considered as pattern recognition problems, etc. "Statistical" refers to the methods used to achieve the task, e.g. neural networks are normally filed under "statistical AI" (for historical reasons). Like I say in my previous comment, modern machine learning research started as a discipline that was separate from pattern recognition (and Pattern Recognition was sometimes considered distinct to AI, as a research subject). I quote below from Tom Mitchell's wildly influential paper, "Generalisation as Search" (AI 18, 1982): "5.2. Statistical pattern recognition The field of statistical pattern recognition deals with one important subclass of generalization problems. In this subclass, the instances are represented by points in n-space, and the generalizations are represented by decision surfaces in n-space (e.g. hyperplanes, polynomials of specified degree). The matching predicate corresponds to determining whether a given point (instance) lies on one side or another of a given decision surface (generalization). The field of Statistical Pattern Recognition has developed very good generalization methods for particular classes of decision surfaces. Many of these methods are relatively insensitive to errors in the data and some have well understood statistical convergence properties, under certain assumptions about the probability distribution of input instances." "In contrast to work in Statistical Pattern Recognition, work on the generalization problem within Artificial Intelligence has focused on problems involving a different class of instance and generalization languages. These languages are incompatible with numerically oriented representations that describe objects as feature vectors in n-space. For example, Winston's program [21] for learning descriptions of simple block structures such as arches and towers, represents instance block structures in terms of their component blocks and relationships among these. In this domain the natural representation for instances is a generalized graph rather than a feature vector. (...) https://www.sciencedirect.com/science/article/abs/pii/0004370282900406 https://www.sciencedirect.com/science/article/abs/pii/000437... Pattern recognition these days is dominated by neural methods, e.g. CNNs for object classification etc. and so is machine learning so it kind of makes sense that the three terms are used interchangeably, but unfortunately many people are not aware of the historical context of the terms, hence misunderstandings as the one by Veedrac above, regarding my comment that machine learning has become a byword for statistical pattern recognition: it's not because I'm dismissive of statistical pattern recognition, or of the ability of deep learning systems to perform it; it's because the terms have really become interchangeable and I think even among researchers. In any case, before commenting on a complex subject of research with a long history, my recommendation remains to first become well acquainted with the subject and its history. Otherwise, one runs the risk of appearing confused. Edit: Note that Mitchell's use of "generalisation" in the excerpt above does not refer to the ability of a model to generalise to test, or unseen, data. Rather, "generalisation" in Mitchell's paper refers to the ability of a system "to take into account a large number of specific observations, then to extract and retain the important common features that characterize classes of these observations." Mitchell's paper describes a theory of search guided by generalisation relations between hypotheses and observations.

deleted 6y ago

[deleted]

YeGoblynQueenne 6y ago

Amicably, can I ask you how you know what you say above to be true? Where does your knowledge of AI come from? Edit: I'm asking because there is evident in your comment a confusion that is all too common on the internets today, of thinking of "GOFAI" (symbolic AI) and "ML" (machine learning) as two somehow incompatible and perhaps exclusive sub-fields of AI. This is as far from the truth as it could be, for instance machine learning really took off as a subject of research in the 1980's with the work of Ryszard Michalski and others, who considered machine learning approaches for the purpose of overcoming the "knowledge acquisition bottleneck", i.e. the difficulty of hand-crafting production rules for expert systems. Indeed, most of the early machine learning systems were propositional-logic based, i.e. symbolic. And of course, one of the most widely used and well-known type of machine learning approach used to this day, decision trees, hail from that time and also learn propositional logic, symbolic models. Of course, most people today know "ML" as a byword for deep learning, or at best statistical pattern recognition (if that). It's just another aberration brought on by the sudden explosion of interest in a once small field. I refer you to Michalski's textbook, "Machine Learning: An Artificial Intelligence Approach" and Tom Mitchell's "Machine Learning" for more information on the early days of the field, i.e. until ca. 2000.

Veedrac 6y ago
It doesn't much matter what people wrongly thought in the past, any more than one has to study the four fundamental elements of water, air, earth and fire to become an expert in chemistry. GOFAI didn't work, hence the winter. If it did I'd think otherwise. It's true that you can, in some places, merge the two. The significant majority of the time, this just makes your ML system worse in the long run, per The Bitter Lesson. Occasionally—very occasionally—your problem is fundamentally simple, so something brittle like AlphaZero works, even though we'd still rather shave the fixed-function parts off with MuZero and the like. But it's no coincidence this is reserved for simple, thoughtless problems (like brute-force move search) and kept isolated from the thinking, as intelligence needs generalization and abstraction, and GOFAI doesn't generalize or abstract. > And of course, one of the most widely used and well-known type of machine learning approach used to this day, decision trees Though decision trees live a healthy life in data analysis, alongside things like k-means clustering, they're obviously not AI. > Of course, most people today know "ML" as a byword for deep learning, or at best statistical pattern recognition (if that). ‘Statistical pattern recognition’ is just name calling devoid of real criticisms. I can ask GPT-3, > a = ["fitness", "health", "heart"], b = ["lifting", "curls", "squats"], c = ["running", "jogging"], so what is b.append("pushups")? and it'll happily answer > b.append("pushups") returns ["lifting", "curls", "squats", "pushups"] or I can give it the Loebner Prize questions which it can almost ace, or I can test it on 10-digit addition and it gets 60% right (with thousands separators, but still suffering BPEs), or I can ask it to do quirky tasks like shuffling letters, or ask it to write sentences with novel words and find that it understands nuanced differences between the definitions... You can scream ‘but it's just statistical pattern recognition’ all you want, but the criticism doesn't mean anything if ‘statistical pattern recognition’ includes this level of generalization, reasoning and algorithmic sophistication in natural language with scaling curves that assure us the best is yet to come.
- YeGoblynQueenne 6y ago
  I "can scream all I want"? Before I continue this conversation, I'd like to ask, do you think this is an appropriate response to my comment? I responded to your original comment respectfully and politely. Note that I did not use "pattern recognition" in a derogatory manner, neither did I say that anything is "just" pattern recognition. I think you may be mistaking my comment for a different opinion that you disagree with.
  - YeGoblynQueenne 6y ago
    While waiting for Veedrac's reply to my comment, I thought I'd clarify here what I meant by "statistical pattern recognition". "Pattern recognition" is the name of the sub-field of AI research from which statistical machine learning grew into what it is today, for example many problems in machine vision are typically considered as pattern recognition problems, etc. "Statistical" refers to the methods used to achieve the task, e.g. neural networks are normally filed under "statistical AI" (for historical reasons). Like I say in my previous comment, modern machine learning research started as a discipline that was separate from pattern recognition (and Pattern Recognition was sometimes considered distinct to AI, as a research subject). I quote below from Tom Mitchell's wildly influential paper, "Generalisation as Search" (AI 18, 1982): "5.2. Statistical pattern recognition The field of statistical pattern recognition deals with one important subclass of generalization problems. In this subclass, the instances are represented by points in n-space, and the generalizations are represented by decision surfaces in n-space (e.g. hyperplanes, polynomials of specified degree). The matching predicate corresponds to determining whether a given point (instance) lies on one side or another of a given decision surface (generalization). The field of Statistical Pattern Recognition has developed very good generalization methods for particular classes of decision surfaces. Many of these methods are relatively insensitive to errors in the data and some have well understood statistical convergence properties, under certain assumptions about the probability distribution of input instances." "In contrast to work in Statistical Pattern Recognition, work on the generalization problem within Artificial Intelligence has focused on problems involving a different class of instance and generalization languages. These languages are incompatible with numerically oriented representations that describe objects as feature vectors in n-space. For example, Winston's program [21] for learning descriptions of simple block structures such as arches and towers, represents instance block structures in terms of their component blocks and relationships among these. In this domain the natural representation for instances is a generalized graph rather than a feature vector. (...) https://www.sciencedirect.com/science/article/abs/pii/0004370282900406 https://www.sciencedirect.com/science/article/abs/pii/000437... Pattern recognition these days is dominated by neural methods, e.g. CNNs for object classification etc. and so is machine learning so it kind of makes sense that the three terms are used interchangeably, but unfortunately many people are not aware of the historical context of the terms, hence misunderstandings as the one by Veedrac above, regarding my comment that machine learning has become a byword for statistical pattern recognition: it's not because I'm dismissive of statistical pattern recognition, or of the ability of deep learning systems to perform it; it's because the terms have really become interchangeable and I think even among researchers. In any case, before commenting on a complex subject of research with a long history, my recommendation remains to first become well acquainted with the subject and its history. Otherwise, one runs the risk of appearing confused. Edit: Note that Mitchell's use of "generalisation" in the excerpt above does not refer to the ability of a model to generalise to test, or unseen, data. Rather, "generalisation" in Mitchell's paper refers to the ability of a system "to take into account a large number of specific observations, then to extract and retain the important common features that characterize classes of these observations." Mitchell's paper describes a theory of search guided by generalisation relations between hypotheses and observations.