10 ms·
The Chinese Room has been on my mind lately after recently learning about it: https://en.m.wikipedia.org/wiki/Chinese_room https://en.m.wikipedia.org/wiki/Chin
by mensetmanusman 5y ago
The Chinese Room has been on my mind lately after recently learning about it:
https://en.m.wikipedia.org/wiki/Chinese_room https://en.m.wikipedia.org/wiki/Chinese_room
Highly recommend trying to grasp the argument, because in a sense it connects complexity to ‘time to compute’. If it takes a room of robots following simple instructions a billion years to act as an AI that reads Chinese, are those robots a collective intelligence? Can it be scaled up to consciousness?
- karpierz 5y agoWhere does computational complexity factor into the Chinese room? Whether the Chinese room responds quickly or slowly doesn't really factor into whether the room understands Chinese.
- Strilanc 5y agoThe linked article discusses this.
- skulk 5y agoWhy stop at a computer, why not extend the argument to a full simulation of a human brain? What if I sat in my room for 3 million years simulating 3 seconds of a human body including the brain? Assuming that my simulation is faithful, who's to say that my simulation doesn't have some form of consciousness or subjective experience?
- AnIdiotOnTheNet 5y agoYou can go even deeper: given that your simulation follows a set of known rules from an initial state, what does actually performing those steps even matter? Is the consciousness given existence because you perform calculations on paper, or does it exist already within the fabric of reality and you're merely exposing it?
- 3np 5y agoI think a common mistake is to conflate “consciousness” with “intelligence” or at least “thought”. Just because we don’t tend to recognize one without the other doesn’t mean they necessarily have a causative relationship.
- tshaddox 5y agoI'm not sure what you mean. Just listing the steps of an algorithm is quite different than executing the steps of an algorithm. As an obvious example, with an algorithm for summing two numbers together, the difference between listing the steps and executing them is that when you execute them you are provided with the sum. I don't see any sense in which these aren't different, unless perhaps you're introducing some separate cosmological "Boltzmann brain"-style argument that all possible finite results of all halting algorithms will arise or have arisen due to random fluctuations. But in that case you've got a whole new set of assumptions to address.
- skulk 5y agoThe idea of a fundamental difference between the result of "performed" and "un-performed" computation always leads me to imagining the null universe, where nothing gets computed ever but all math "exists" as we'd expect. It begs the question, why is the universe non-null? Allowing the mere existence of the possibility of computation to answer that question is very comforting. But yes, it requires approximately as much faith as any monotheistic religion.
- AnIdiotOnTheNet 5y agoDoes it really? When one considers that we take seriously the many worlds interpretation of quantum mechanics and nigh-uncountable Calabi-Yau manifold topologies of superstring theory? There is some suggestion that our experiential reality is but one infinitesimal slice of existence even among the empirically minded.
- skulk 5y ago
- Der_Einzige 5y agoThis is why I find it so strange that philosophers seem to by in large accept "Cogito Ergo Sum" or "I think therefor I am" as a valid proof of ontology. What if the appearance of thought is not actually thought? Did philosophers not watch Blade runner? Sure Descartes didn't - but contemporaries sure as shit do!
- cuspycode 5y agoBut how do you even distinguish appearance of thought from actual thought, objectively? Of course you can always argue that actual thought is something that is subjectively experienced, but that leads to solipsism, doesn't it?
- Der_Einzige 5y agoSolipsism is what you get from accepting the Cogito "I know I exist for sure, and only my own existence can be verified by the cogito" To answer your question, I claim that you can't. Radical skepticism ("we live in the universe of the evil demon"), which is what I advocate for, means that you can't even claim that you exist. See the works of Max Stirner for further elaboration of the implications of this for Philosophy.
- jean_tta 5y agoThis is essentially the philosophical zombie thought experiment: https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie
- shadowgovt 5y agoNot to downplay Descartes (his philosophy, specifically the process he undertakes, is worth a study), but something that the pop summaries of his writing really gloss over is that his entire premise hinges on something he takes as assumed true that need not be, and if it isn't, the rest goes straight off the rails. Paraphrasing from memory: he pins "cogito ergo sum" on the assumption that objective knowledge is possible. That assumption is built atop imagining an alternative: an evil demon has full control over his senses and feeds him whatever it wants to (essentially, the 'Matrix hypothesis'). His solution to this concern? "A loving God wouldn't create a universe that banal and meaningless." It's only a proof if you accept the postulate that objective reality is correlated with experience.
- arketyp 5y agoYeah, this is related to Stephen Wolfram's principle of computational equivalence and his re-take on "the weather has a mind of his own". Turing machines occur everywhere in nature, emerge from the simplest rules, Wolfram demonstrates, so maybe consciousness is everywhere too. Where to put the scope of computing processes basically. Searle refutes it saying it's nonsensical to contend that a concrete wall could be conscious. I don't know, who knows.
- np- 5y agoIt doesn't seem particularly nonsensical to contend a concrete wall has some form of consciousness. Concrete walls don't spring up out of nowhere, they're built by humans who have a need to separate things (either for good or for bad). A single concrete wall might not seem like much, but in the greater scope of things, the rising and falling of all concrete walls seems to represent some sort of plane of humanity and its consciousness. Even think of the extremely complicated emotions evoked by different types of walls: i.e. the Berlin Wall, the U.S.-Mexico Wall, the Great Wall, etc. I know I'm probably reading way too much into this specific example, but I think this could generally apply to a lot of things.
- Yajirobe 5y ago> If it takes a room of robots following simple instructions a billion years to act as an AI that reads Chinese, are those robots a collective intelligence? Can it be scaled up to consciousness? No, because of the article you linked to. Chinese room argues AGAINST the existence of consciousness/intelligence arising from following symbolic instructions. Here is a quote I like by John Searle: > Computational models of consciousness are not sufficient by themselves for consciousness. The computational model for consciousness stands to consciousness in the same way the computational model of anything stands to the domain being modelled. Nobody supposes that the computational model of rainstorms in London will leave us all wet. But they make the mistake of supposing that the computational model of consciousness is somehow conscious. It is the same mistake in both cases.
- hackinthebochs 5y agoSearle's argument misses the mark. A simulated rainstorm wont leave us wet, but a simulation of a disordered state is (or contains) a disordered state. If consciousness is a relational property of a system, then a simulation of such a relational property will instantiate that relational property without qualification. The Chinese room argument can't address this conception of consciousness.
- jimbokun 5y ago> If consciousness is a relational property of a system But first you have to establish this is true.
- hackinthebochs 5y agoNot really. The point is that as an argument against computationalism about the mind, Searle's Chinese room can't rule out computationalism if mind is merely a relational property of a system.
- mjburgess 5y agoThe argument is against this view of consciousness. More intuitively: our words are things which do things in the world. When we acquire them we do so by being embedded in the world. When we use them we use them to change the world. (Largely). If a child in a cupboard reading a textbook out-loud could fool a chemist that the child understood chemistry, it doesn't mean they do. In the end, you need to turn on that bunsen burner. The chinese room needs to be taken super-literally: does the person just doing a "hash-table lookup" on "the right answers to pre-prosed questions" actually understand. To me its incredibly obvious that they do not. This does not establish that, eg. cognition, does not involve computational hastable-like processes. I think it does establish that propontents of a naive computationalism have an incredibly big challenge. The challenge is to specify the environment computationally, since that is the only means by which the computer itself can be sufficiently embedded and actually understand anything. The issue here is that if "computer" isnt just a trivial property it has to include, eg., measurability, determinism, etc. Properties that reality lacks. Reality isnt a computer. Starting with the chinese room one /can/ supplement sufficiently until we reach searle's conclusion.
- dragonwriter 5y ago> Highly recommend trying to grasp the argument It's not really, IMO, all that valuable. > because in a sense it connects complexity to ‘time to compute’. No, it doesn’t. It basically just rejects by definition (and then presents an illustration which is a useful support if you accept its definition or an illustration that its definitions are wrong if you don’t) that a programmed response system that exhibits the behavior of an actor that understands material actually understands material, and from that rejects the idea that brains (which produce minds, which have actual understanding) are equivalent to programmed response systems.
- pdabbadabba 5y ago> No, it doesn’t. It basically just rejects by definition... Its been a little while since I've read the paper, but I don't think it's right to say that it rejects this conclusion by definition. Instead, it presents a situation where the reader's intuitions are intended to rebel at the thought that the system in question understands the symbols that it is manipulating. But yes, otherwise I agree with you that the argument then goes that, if this case has caused you to be skeptical about whether a symbol manipulation system could be conscious, you should be skeptical about whether other ones would be as well. There are, of course, some important limitations/objections. Two that come to mind are that 1) you might not think that the reader's intuitions about a situation have any metaphysical significance--they might just be wrong and 2) the Chinese Room example might just suggest that a particular kind of symbol manipulation system may not be conscious without supporting the much stronger claim that they can't be conscious. The fact that one can raise objections, though, doesn't mean that it isn't an important contribution to the philosophy of mind. If nothing else, I think Searle was right to point out that the example raises interesting questions about the usefulness of the Turing Test since, by hypothesis, the Chinese Room could pass it.
- codeulike 5y agoIf nothing else, I think Searle was right to point out that the example raises interesting questions about the usefulness of the Turing Test since, by hypothesis, the Chinese Room could pass it. The other part of the thought experiment, apart from imagining the room, is to imagine how complex the turing test might be. When people imagine what a Turing Test conversation would look like, they frequently underestimate the conversation. I find Dennet's example of an imaginary Turing Test from Consciousness Explained to be a good counterexample: Judge: Did you hear about the Irishman who found a magic lamp? When he rubbed it a genie appeared and granted him three wishes. “I’ll have a pint of Guiness!” the Irishman replied and immediately it appeared. The Irishman eagerly set to sipping and then gulping, but the level of Guiness in the glass was always magically restored. After a while the genie became impatient. “Well, what about your second wish?” he asked. Replied the Irishman between gulps, “Oh well, I guess I’ll have another one of these.” CHINESE ROOM: Very funny. No, I hadn’t heard it– but you know I find ethnic jokes in bad taste. I laughed in spite of myself, but really, I think you should find other topics for us to discuss. J: Fair enough but I told you the joke because I want you to explain it to me. CR: Boring! You should never explain jokes. J: Nevertheless, this is my test question. Can you explain to me how and why the joke “works”? CR: If you insist. You see, it depends on the assumption that the magically refilling glass will go on refilling forever, so the Irishman has all the stout he can ever drink. So he hardly has a reason for wanting a duplicate but he is so stupid (that’s the part I object to) or so besotted by the alcohol that he doesn’t recognize this, and so, unthinkingly endorsing his delight with his first wish come true, he asks for seconds. These background assumptions aren’t true, of course, but just part of the ambient lore of joke-telling, in which we suspend our disbelief in magic and so forth. By the way we could imagine a somewhat labored continuation in which the Irishman turned out to be “right” in his second wish after all, perhaps he’s planning to throw a big party and one glass won’t refill fast enough to satisfy all his thirsty guests (and it’s no use saving it up in advance– we all know how stale stout loses its taste). We tend not to think of such complications which is part of the explanation of why jokes work. Is that enough? Dennett: “The fact is that any program that could actually hold up its end in the conversation depicted would have to be an extraordinary supple, sophisticated, and multilayered system, brimming with “world knowledge” and meta-knowledge and meta-meta-knowledge about its own responses, the likely responses of its interlocutor, and much, much more…. Maybe the billions of actions of all those highly structured parts produce genuine understanding in the system after all.”
- kordlessagain 5y ago> To all of the questions that the person asks, it makes appropriate responses, such that any Chinese speaker would be convinced that they are talking to another Chinese-speaking human being. This is a fallacy, regardless of the language. Some comms between conscious entities must always contain metaphoric language, including signaling through body language. A shrug is a metaphor for uncertainty. This is why the premise of the Chinese Room is true (we can’t build a mind in a box) but also why conscious machines will necessarily have to have bodies. Anything that is conscious and can hold a (real) dialog with a human will have emotional output tied to the body and it’s collective experience.
- michaelmrose 5y agoWhy couldn't you simulate the body too?
- tshaddox 5y ago> Some comms between conscious entities must always contain metaphoric language, including signaling through body language. What do you mean? Humans can certainly communicate and maintain meaningful relationships solely through writing.
- tsimionescu 5y agoAre we not communicating right now without seeing each other's bodies?
- achr2 5y agoIt seems to me that the argument is kind of silly. The "Chinese Room" has failed to account for the actual processes of the system. The 'spoken response' is only a small part of the 'program' of a mind; the rest is the internal processes that constitute consciousness that can infer, compare, reinforce, and associate. Consciousness is the whole of the system, not a functional IO.
- new_guy 5y agoIt's basically a red-herring (fallacious argument), the robots/workers etc are just homunculus, and the entire thing falls apart without them.
- deleted 5y ago[deleted]
- PeterisP 5y agoIMHO the Chinese room thought experiment is essentially a tautology which hinges on the validity of the key assumptions it implies, namely, that such a room is possible. So, Searle supposes that he is in a closed room and has a book with encoded instructions that supposedly allow him to communicate in Chinese; and he also supposes that being able to convince the Chinese-speaker that they're speaking to a human (i.e. passing a Turing test) is sufficient to demonstrate "understanding" - for whatever definition of "understanding" Searle wants to use. If we discount the second assumption (which might be the case - IMHO humans could be tricked in a Turing test by something that makes zero pretense at understanding anything), then there's no paradox, then it's just an experiment that's too weak to say anything about "understanding" whatsoever; but since Searle considers the experiment valid, let's assume that this holds, and if someone/something can convince the Chinese person of "proper understanding", then that someone/something would really have "proper understanding". So it comes down to whether simply blindly following instructions a book can achieve proper understanding. If yes, then Searle must concede the argument. And if not, then Searle's room simply doesn't work - apparently such a book is impossible to make or have, the Chinese room can't exist and the impossible experiment can't have any results. In essence, it's a tautology - no matter how ridiculous you consider encoding "proper understanding" in a book of instructions, if you start with an assumption that you do have such a book, then you obviously have assumed a world where the combination of instructions+mindless executor can have "proper understanding"; and if you start with an assumption that a machine like that definitely can not be thinking "properly" (as Searle does) and Strong AI is impossible, then you have thrown away the key thing required for the thought experiment - the possibility for such a Strong-AI-enabling book to exist. If you make two assumptions and find out that they are incompatible - like Searle does here - then you can deduce that at least one of them is wrong, but it does not say anything at all about which one of them is wrong, so I don't see much value in this thought experiment.
- jltsiren 5y agoPhilosophical confusion often ensues when we use words outside their context. In this case, words like "intelligence", "consciousness", and "understanding" are meaningless, because they have no widely understood meanings in the context of non-human systems. The Chinese room argument is effectively a counterargument to the Turing test. Turing defined intelligence in terms of external behavior of the system. According to him, a system is intelligent, if its behavior is indistinguishable from that of a human in a task that requires intelligence. Searle claimed that intelligence is a property of the physical system itself, not of its external behavior. In order to determine whether a system understands something, we have to observe its internal behavior. A state machine could in principle pass the Turing test due to an extensive set of rules, but Searle would not call it intelligent, because it is fundamentally a simple mechanism. The original Chinese room argument went further. Searle claimed that the same principle that rules out the intelligence of a state machine also rules out the intelligence of any programmed computer. I think that can be attributed to taking models of computation too literally. If we define computation as taking a fixed program and a single input and producing a single output, it feels mechanical rather than intelligent. A system like that can simulate another system that interacts with the environment and updates its internal state, but like Searle himself claimed, a simulation is not the same as the simulated system.
- AtlasBarfed 5y agoYuck, I didn't like that argument at all. The point about him doing the manual execution of the computer is a stupid parlor trick. Individual proteins in axons have no special awareness of "English" encoded in the firing. RNA expression has no special awareness of "English". It seems like appeal to bias rather than an actual argument. I find far too many philosophical constructs insufficiently deconstructed. Plus everyone has a different personal interpretation of language and meaning shaded by different exposures and interpretations of sensory input. Sufficient common experience is likely what gives words some semblance of common meaning between people. I would argue what makes AI truly hard to do is that a computer can't have the experience of birth, child rearing, etc. It's too foreign. But at some point language veers into a coded representation of physical laws and reality, and there AI can converge with human experience.
- rowanG077 5y agoCan't this argument be pretty easily countered with quantum mechanics and randomness induced by it? I'm no physicist but if you have a quantum random source of bits the computer and the person would not reach the same answer. It could then be argued consciousness arises from the quantum nature of the universe.
- tsimionescu 5y agoBy the same token, if you have a quantum random source of bits two identical computers running the same computation but using that source as part of the input will not always reach the same answer. Does that prove that they are running different computations? Does your CPU become conscious when a cosmic ray interacts with its RAM (a rare but far from unheard of occurence of quantum noise affecting all computers on Earth, much more common for computers outside the atmosphere).