Philosophical confusion often ensues when we use words outside their context. In this case, words like "intelligence", "consciousness", and "understanding" are

Philosophical confusion often ensues when we use words outside their context. In this case, words like "intelligence", "consciousness", and "understanding" are meaningless, because they have no widely understood meanings in the context of non-human systems. The Chinese room argument is effectively a counterargument to the Turing test. Turing defined intelligence in terms of external behavior of the system. According to him, a system is intelligent, if its behavior is indistinguishable from that of a human in a task that requires intelligence. Searle claimed that intelligence is a property of the physical system itself, not of its external behavior. In order to determine whether a system understands something, we have to observe its internal behavior. A state machine could in principle pass the Turing test due to an extensive set of rules, but Searle would not call it intelligent, because it is fundamentally a simple mechanism. The original Chinese room argument went further. Searle claimed that the same principle that rules out the intelligence of a state machine also rules out the intelligence of any programmed computer. I think that can be attributed to taking models of computation too literally. If we define computation as taking a fixed program and a single input and producing a single output, it feels mechanical rather than intelligent. A system like that can simulate another system that interacts with the environment and updates its internal state, but like Searle himself claimed, a simulation is not the same as the simulated system.