4 ms·
#2 is explored by John Searle's Chinese Room argument, which is unfortunately completely misunderstood by most technologists working in AI, to their own peril.
by keiferski 7y ago
#2 is explored by John Searle's Chinese Room argument, which is unfortunately completely misunderstood by most technologists working in AI, to their own peril.
This article is an excellent introduction:
http://www.dreamsongs.com/Searle.html http://www.dreamsongs.com/Searle.html
A relevant excerpt:
Searle goes on to contend that intentional behavior must be in response to real contingencies, not formal simulations of reality. If so, it would seem to follow that the capacity for intentional behavior is the result of what might be called phylogenic contingencies, and so would only be found in products of real evolution. In short, what Searle is saying is that, just like other biological activities such as digestion and photosynthesis, thought is intrinsically dependent on the biochemistry of its origin. Just as a formal simulation of digestion is not really digesting, a formal simulation of thought is not really thinking. And to tie this to the gender thought experiment, a formal simulation of gender is not gender, because it lacks biological structural and functional characteristics.
The "Simulation, Duplication and Evolution" section of the SEP article is also useful:
https://plato.stanford.edu/entries/chinese-room/#SimuDuplEvol https://plato.stanford.edu/entries/chinese-room/#SimuDuplEvo...
- gowld 7y agoSearle either misunderstood AI (and computers) or explained himself poorly (as philosophers often do), so misunderstanding Searle isn't a major problem.
- mannykannot 7y agoI seriously doubt that Searle misunderstands computers, and probably not what passes for AI either then or now. I also regard that paper as one of the more accessible works of the genre, though, as always, you have to pay close attention to what claims are being made. On the other hand, at least at the time of writing the original paper, he clearly misunderstood the 'systems reply', as his response to it -- supposing that the actor had memorized the instructions -- is beside the point, and makes no difference.
- lidHanteyk 7y agoOne important nuance to the Chinese Room is to keep in mind that, even as a black box, there must be limitations on its behavior. We can infer those limitations. We start with Aaronson's observation [0] that the Chinese Room's complexity can be analyzed. Let us assume that the Room responds in (N)P time, and in particular not EXP time. Then the dictionary must be poly-sized. Further, the dictionary is finite but the input strings are unbounded in length, so that we must be able to divide and conquer somehow. Therefore we only need log-many steps to walk the dictionary. The dictionary is ordered, and so indexed, and so walked in log-time. We can conclude that Searle's definition of intelligence is basically the ability to execute certain kinds of basic katamorphisms, akin to pregroup grammars [1] or other categorical grammars; in other words, Searle says that intelligence is merely the ability to talk convincingly like a human, and not necessarily any of the other features that we see across Mammalia. Searle might retort that the Room knows facts about the world. This would require us to augment our pregroup/category, enriching it with relational information. For example, in the utterance "I like cats", we might annotate each word with a relation or type, and then look them up in a secondary database-book. We can see here that it would then be the database-book which has an opinion on cats, since "I" becomes merely another symbol in the database. This is reminiscent of Hofstadter's critique of Searle (supported by his other work), in which he suggests that "I" is indeed merely a self-referent symbol which leads to a strange loop, or as he puts it, "I am a strange loop." Here, the strangeness of the loop is provided by the ability of "I" to be both a plain word in the textual language and also a referent symbol in the symbolic database. Edit: Forgot the main takeaway: When Searle and Turing measure intelligence based on a person's ability to respond to textual queries with textual answers, but the entire interface between the tester and the person is the ability to give textual answers to textual questions, then we aren't measuring intelligence anymore. We've struck Goodhart's Law [2] and we can imagine that the most successful people in these tests will be those who can abuse glitches in text-processing systems. [0] https://www.scottaaronson.com/papers/philos.pdf https://www.scottaaronson.com/papers/philos.pdf [1] https://en.wikipedia.org/wiki/Pregroup_grammar https://en.wikipedia.org/wiki/Pregroup_grammar [2] https://en.wikipedia.org/wiki/Goodhart's_law https://en.wikipedia.org/wiki/Goodhart's_law
- mannykannot 7y ago> Searle might retort that the Room knows facts about the world. IIRC, Searle's claims to show that the instruction book could not be written, and therefore the room could not exist, so he presumably does not attribute any properties or capabilities to it.
- erostrate 7y agoSee Mike Adler answer to Searle. To use your digestion metaphor, as long as the input is food and the output is energy and byproducts, AI researchers don't care whether philosophers would call it "real digestion" or a "formal simulation of digestion". Same goes for thinking. https://philosophynow.org/issues/46/Newtons_Flaming_Laser_Sword https://philosophynow.org/issues/46/Newtons_Flaming_Laser_Sw...
- keiferski 7y agoThis essay is filled with so many falsehoods, assumptions, and lack of knowledge that I don't know where to begin. It's also written with a snarky, arrogant attitude which is not helpful. To list a few: - "there is every reason to believe that a human brain is a machine" Given what evidence? - Platonism died for mathematicians some centuries ago, and simply looks silly. And yet Frege and Gödel, two of the most accomplished mathematicians of the past century, could be described as Platonists. Clearly this isn't a closed question. https://en.wikipedia.org/wiki/Philosophy_of_mathematics#Platonism https://en.wikipedia.org/wiki/Philosophy_of_mathematics#Plat... https://plato.stanford.edu/entries/platonism-mathematics/ https://plato.stanford.edu/entries/platonism-mathematics/ Regarding digestion: if the Turing Test is determining whether a computer is thinking like a human being, then focusing only on input and output, and not the 'being', sort of misses the entire point. Remember that Searle's Chinese Room is primarily a criticism of the Turing Test. As always, the truth is: it's more complicated than it seems.
- erostrate 7y agoI certainly agree that Adler's provocative tone is not helpful and his strokes are sometimes too broad to be fully correct as you have rightfully pointed out, but the general point stands. To me the Turing test gives a falsifiable experiment that defines what "thinking like a human being" means. It does so by focusing only on the input output. To do that, it puts the system under test out of view of the experimenter and mediates the experiment with written text. So yes, the input output is the important part, the Turing test, albeit flawed, is interesting, and any claim that machines can't think based on a definition of "think" that does not lend itself to a falsifiable experiment, such as Searle's, is uninteresting. I will leave it at that and make a quick exit, as Adler suggests.
- ericjang 7y agoI agree with Searle insomuch that Life and Death is the real contingency with which gives a reality meaning. If the simulation had some arbitrary cellular-automata ruleset, there is no guarantee that the simulation would ever evolve into something isomorphic to our reality. I have a strong belief that 3D physics + collision/contact forces, conservation of energy, linearity of perspective, and a nutrient cycle are important pieces. If thought is indeed "intrinsically dependent on biochemistry...", then we have a hard journey ahead. However, AI philosophers should try their own medicine and present claims with real contingencies, i.e. make claims that are experimentally falsifiable. Put another way, I'm less interested in debating what "understanding" is and more interested in figuring out how to build a Chinese room to begin with. Once we have such a system, perhaps we might realize that the Chinese Room thought experiment was ill-posed to begin with.
- mannykannot 7y ago> Searle goes on to contend that intentional behavior must be in response to real contingencies, not formal simulations of reality. If so... He does indeed, but the Chinese Room paper presents no compelling argument for that view; it ultimately depends on an appeal to one's intuitions about the feasibility of a remarkable and unrealistic situation that is far from our own experiences, and one, moreover, where he misunderstood the most common response (the 'systems reply'). Dennett has objected that these sort of arguments (or, as he calls them, 'intuition pumps') are simply unsuitable for proving anything, comparing them to denying heliocentricity on the grounds of it being just obvious that the Earth is not moving.