5 ms·
While waiting for Veedrac's reply to my comment, I thought I'd clarify here what I meant by "statistical pattern recognition". "Pattern recognition" is the name
by YeGoblynQueenne 6y ago
While waiting for Veedrac's reply to my comment, I thought I'd clarify here what I meant by "statistical pattern recognition". "Pattern recognition" is the name of the sub-field of AI research from which statistical machine learning grew into what it is today, for example many problems in machine vision are typically considered as pattern recognition problems, etc. "Statistical" refers to the methods used to achieve the task, e.g. neural networks are normally filed under "statistical AI" (for historical reasons).
Like I say in my previous comment, modern machine learning research started as a discipline that was separate from pattern recognition (and Pattern Recognition was sometimes considered distinct to AI, as a research subject). I quote below from Tom Mitchell's wildly influential paper, "Generalisation as Search" (AI 18, 1982):
"5.2. Statistical pattern recognition
The field of statistical pattern recognition deals with one important subclass of generalization problems. In this subclass, the instances are represented by points in n-space, and the generalizations are represented by decision surfaces in n-space (e.g. hyperplanes, polynomials of specified degree). The matching predicate corresponds to determining whether a given point (instance) lies on one side or another of a given decision surface (generalization). The field of Statistical Pattern Recognition has developed very good generalization methods for particular classes of decision surfaces. Many of these methods are relatively insensitive to errors in the data and some have well understood statistical convergence properties, under certain assumptions about the probability distribution of input instances."
"In contrast to work in Statistical Pattern Recognition, work on the generalization problem within Artificial Intelligence has focused on problems involving a different class of instance and generalization languages. These languages are incompatible with numerically oriented representations that describe objects as feature vectors in n-space. For example, Winston's program [21] for learning descriptions of simple block structures such as arches and towers, represents instance block structures in terms of their component blocks and relationships among these. In this domain the natural representation for instances is a generalized graph rather than a feature vector. (...)
https://www.sciencedirect.com/science/article/abs/pii/0004370282900406 https://www.sciencedirect.com/science/article/abs/pii/000437...
Pattern recognition these days is dominated by neural methods, e.g. CNNs for object classification etc. and so is machine learning so it kind of makes sense that the three terms are used interchangeably, but unfortunately many people are not aware of the historical context of the terms, hence misunderstandings as the one by Veedrac above, regarding my comment that machine learning has become a byword for statistical pattern recognition: it's not because I'm dismissive of statistical pattern recognition, or of the ability of deep learning systems to perform it; it's because the terms have really become interchangeable and I think even among researchers.
In any case, before commenting on a complex subject of research with a long history, my recommendation remains to first become well acquainted with the subject and its history. Otherwise, one runs the risk of appearing confused.
Edit: Note that Mitchell's use of "generalisation" in the excerpt above does not refer to the ability of a model to generalise to test, or unseen, data. Rather, "generalisation" in Mitchell's paper refers to the ability of a system "to take into account a large number of specific observations, then to extract and retain the important common features that characterize classes of these observations." Mitchell's paper describes a theory of search guided by generalisation relations between hypotheses and observations.
- Veedrac 6y ago> Note that I did not use "pattern recognition" in a derogatory manner, neither did I say that anything is "just" pattern recognition. I think you may be mistaking my comment for a different opinion that you disagree with. > [...] > In any case, before commenting on a complex subject of research with a long history, my recommendation remains to first become well acquainted with the subject and its history. Otherwise, one runs the risk of appearing confused. Sorry, I did misunderstand what you were saying. I do find it somewhat baffling how you're talking as if 1980s terminology is today's terminology, or that everyone should just recognize it as the default way of things. As much as you argue I should adopt the subject's history, I'm saying you should adopt its present. The point you're making would have held fine two decades ago, but as a description of modern ML, it's like calling a sedan a cart. Even if you don't mean it as a criticism (‘just a cart without a horse’), the world has moved on.
- YeGoblynQueenne 6y ago>> Sorry, I did misunderstand what you were saying. Thank you for your honesty and there's no need to apologise. Let me explain my point above, if I can. You said that search, planning, expert systems, etc, are "not AI, and they never will be". I understand that as saying that such systems are not artificial intelligences, in the sense of an intelligent machine created by humans out of whole cloth (without trying to define what "intelligent" means). That is certainly true, but it is also uncontroversial that the above are sub-fields of the field of research that is known as "AI". That is, there is a field that researches approaches that could lead to the creation of intelligent machines and that is called "AI" and then there's the ultimate target of that field which is to create "AI". My original comment bemoans the fact that in some sectors, "AI", as a field of research, has become synonymous with only one sub-sub-field of AI research, that is, deep learning. Contrary to what you state, these "GOFAI" fields (symbolic AI, if you will) are still active and far from having "failed" in any way, they are "SOTA" in their respective tasks. For example, the field of automated theorem proving is dominated by systems that employ the resolution rule, a logic-based approach and while recent efforts have been made to make inroads into theorem proving with deep neural nets (e.g. a recent attempt to use transformers) results are still very far from being comparable to the traditional approaches. I know more about automated theorem proving that I know e.g. about planning or search (because my PhD is in a subject close to the former) but my understanding is that in those fields too, traditional techniques dominate- which is why research in them is still active. If I am permitted to tout my own horn, my broader subject area can be described as "program learning", i.e. machine learning of arbitrary programs from examples of their inputs and outputs. In this area also, deep learning systems are hopelessly outclassed by symbolic approaches, not least because these approaches learn precise representations of target programs (rather than approximations) from a mere handful of examples (four or five, etc). And so on. These are just some examples of AI research that is ongoing, that is not deep learning, and that is state of the art for its respective tasks. In view of that, I consider that using "AI" to mean "deep learning" (as the article above does) is not only misinformed but also misinforming and actively harmful. In the sense of harming people's understanding, that is. As to your comment about how GOFAI "failed", I'm afraid this opinion, which is common, is also misinformed. Here, too, a knowledge of the history of the field helps, but to summarise, the last winter happened because of strictly political reasons and for no reason that had anything to do with the scientific merits, or practical successes of the relevant approaches. In fact, expert systems, now widely considered a failure, were one of the first big success stories of AI; a success story that was cut short only because, again, of political reasons. I could talk about the AI winter subject for hours (it's a favourit subject of mine) but a good starting point is this article by the editor of the IEEE journal Intelligent Systems: (Avoiding Another Winter) https://www.computer.org/csdl/magazine/ex/2008/02/mex2008020002/13rRUyeCkdP https://www.computer.org/csdl/magazine/ex/2008/02/mex2008020.... The wikipedia page on https://en.wikipedia.org/wiki/AI_winter https://en.wikipedia.org/wiki/AI_winter also has a decent summary and some sources. Finally, see the wikipedia article on the Lighthill Report https://en.wikipedia.org/wiki/Lighthill_report https://en.wikipedia.org/wiki/Lighthill_report which is more relevant to AI research in the UK (the Report killed AI research in the UK, dead) and this review of the report by John McCarthy: http://www-formal.stanford.edu/jmc/reviews/lighthill/lighthill.html http://www-formal.stanford.edu/jmc/reviews/lighthill/lighthi... (man who coined the term AI and also created LISP on the side). >> As much as you argue I should adopt the subject's history, I'm saying you should adopt its present. More to the point, I'm recommending that you should know a subject's history before forming a strong opinion about its present and future. As to me, I'm firmly rooted to the present. About half of the literature I read is neural networks- and that's not even the subject of my research. But if you think about it, in the era where the trend is to use deep learning, the most interesting results can only come from not using deep learning. In a gold rush, if everyone is digging up Widow's Creek, then Widow's Creek is the last place to dig for gold.