5 ms·
From the paper: >> In this paper, we present an attempt at an architecture which operates on an explicit higher-level semantic representation, which we name a
by YeGoblynQueenne 2y ago
From the paper:
>> In this paper, we present an attempt at an architecture which operates on an explicit higher-level semantic representation, which we name a “concept”.
I wonder if the many authors of the paper know that what they call "concept" is what all of machine learning and AI has also called a "concept" for many decades, and not a new thing that they have just named from scratch.
For instance, classes of "concepts" are the target of learning in Leslie Valiant's "A Theory of the Learnable", the paper that introduced Probably Approximately Correct Learning (PAC-Learning). Quoting from its abstract:
ABSTRACT: Humans appear to be able to learn new
concepts without needing to be programmed explicitly in
any conventional sense. In this paper we regard learning as
the phenomenon of knowledge acquisition in the absence of
explicit programming. We give a precise methodology for
studying this phenomenon from a computational viewpoint.
It consists of choosing an appropriate information gathering
mechanism, the learning protocol, and exploring the class of
concepts that can be learned using it in a reasonable
(polynomial) number of steps. Although inherent algorithmic
complexity appears to set serious limits to the range of
concepts that can be learned, we show that there are some
important nontrivial classes of propositional concepts that
can be learned in a realistic sense
From: https://web.mit.edu/6.435/www/Valiant84.pdf https://web.mit.edu/6.435/www/Valiant84.pdf
Or take this Introduction to Chapter 2 in Tom Mitchell's "Machine Learning" (the original ML textbook, published 1997):
This chapter considers concept learning: acquiring the definition of
a general category given a sample of positive and negative training
examples of the category.
From: https://www.cs.cmu.edu/~tom/mlbook.html https://www.cs.cmu.edu/~tom/mlbook.html (clink link in "the book").
I mean I really wonder some times what is going on here. There's been decades of research in AI and machine learning but recently papers look like their authors have landed in an undiscovered country and are having to invent everything from scratch. That's not good. There are pitfalls that all the previous generations have explored thoroughly by falling in them time and again. Those who don't remember those lessons will have to find that out the hard way.
- mdp2021 2y agoI am not sure that fits the point, YGQ: it seems to me the concept of «concept» in the paper is "the embedding vector we get in systems like SONAR (which we could use to generalize ordered sets of tokens into more complex ideas)". That's pretty specific, only marginally related to past handling as mentioned.
- YeGoblynQueenne 2y agoThat's only the representation of a concept. Different systems and different approaches will have different representations but that doesn't change the fact of what is being represented.
- mdp2021 2y agoBut if the issue is about "research in AI has had to deal with the concept of "concept" since the inception" (and of course it had to), the contribution in this paper is to try an operational implementation that could bear fruit and possibly fix architectural shortcomings of the mainstream effort. (It is not separate from the context of LLMs.)
- YeGoblynQueenne 2y agoRight, but there's been many operationalisations before. That's what's not new. Tome Mitchell's textbook has plenty of examples. Basically all of machine learning is about learning concepts- in practice as well as in theory. That's the whole point.