4 ms·

At some point we’re going to start asking the same questions about humans. Do humans really understand what we read, or have we just picked up weird tricks that

At some point we’re going to start asking the same questions about humans. Do humans really understand what we read, or have we just picked up weird tricks that happen to work? (When you read a novel, the experience of reading the novel is different for everyone. It’s like being on a rollercoaster or amusement park ride. Everyone has roughly the same stimulus but a different mental landscape and a different reaction. Our response to the book is largely a response to our own experience of reading the book, and only partly a response to the content of the book itself. This distinction isn’t just sophistry. Are we prepared for machines that react to books in ways that are completely alien to our own reactions?)

deleted 7y ago
[deleted]
tgv 7y ago
Certainly a lot of exams are filled in using tricks, bordering on superstition. Words that remind a student of a certain kind of solution can cause application of the wrong formula. In that sense, ML, statistical, or "deep learning" models perform a bit like us. Not surprisingly, since co-occurrence plays a large role in our fast intuitions. It also shows that such models are miles away from higher level reasoning.
pmoriarty 7y ago
"At some point we're going to start asking the same questions about humans. Do humans really understand what we read, or have we just picked up weird tricks that happen to work?" That question has been asked by philosophers and psychologists for a long time. There's a branch of philosophy called epistemology -- the study of how we know what we know -- where questions about whether we even know what we claim to know (or understand what we claim to understand) belong. In addition, nearly a hundred years ago behaviorist psychologists were positing that all we can see is other people's behavior, so making claims about their supposed mind states, understanding, etc, were unwarranted and effectively off-limits... and then psychology, cognitive science and linguistics had a backlash against behaviorists, rejecting their approach and assumptions, with mental states, thought, and understanding being admissible in the field for many decades now. So none of this is really new, and mountains of books and papers have been written discussing and debating such issues for at least a hundred years. The Chinese Room thought experiment discussed in the article has been a staple of many Philosophy 101 courses, where the obvious followup question to ask (even for students just getting their first exposure to philosophy) is whether humans understand what they read to begin with. Pick up any related philosophy book and they'll discuss such questions ad nauseum. Taking a much further step back in to history, more than 2000 years ago, the Skeptics were questioning whether we can know anything, and some were denying that anything can be known. Philosophers in the West have been debating such questions ever since. In Eastern philosophy and religion, such concerns are also very old, with a variety of people asserting the illusory nature of worldly knowledge, and positing that true reality is unknowable, or at least not accessible or limited by rational thought. Similar themes are echoed in Western mysticism. Jumping forward yet again (forgive the rambling nature of this response), we have Descartes Duck[1], and the popular Enlightenment and Post-Enlightenment view that "man is just a machine". Questions about whether we really understand or are really doing no more than a sophisticated version of what the Chinese Room or "dumb" neural networks do could be seen as modern restatements of that view. [1] - https://en.wikipedia.org/wiki/Digesting_Duck https://en.wikipedia.org/wiki/Digesting_Duck
- klodolph 7y ago
  The Chinese room thought experiment is based on logically unsound reasoning and I don’t understand why it has taken hold as widely as it has. The logical argument is that a composite object of a book, room, and person following directions cannot understand Chinese, but there is no justification for this claim—it is, as far as I can tell, a complete non-sequitur. Perhaps I just misunderstand it? Because the logic underlying the Chinese room argument is unsound, it naturally leads to some bizarre conclusions. I think it’s a bit of a tragedy that it’s a staple of philosophy 101 courses, unless you use it as an example for teaching logic by dissecting it and showing how it is unsound. Personally my view is that as much as we need our scientists and ML researchers to study epistemology, we need our epistemologists to study science and mathematics. Any coherent and comprehensive epistemological system in the future will have to somehow synthesize things like substance monism, post-Fisherian / post-Bayesian statistics, post-Popper science, and computational complexity theory. Just as too many ML researchers are reluctant to dip into philosophy and reinvent a poorer version of old philosohpical arguments, there are too many philosophers that would rather e.g. rehash some ideas of Hegel rather than step outside their own field long enough to synthesize new ideas.
  - jowday 7y ago
    I never interpreted it as implying the 'composite object of a book, room, and the person following directions' can't understand Chinese, but rather that the person doesn't understand Chinese.
  - kthejoker2 7y ago
    Yes, you misunderstand it. The Chinese Room is a refutation of the Turing Test and similar claims on AGI. It has nothing to do with the person in the room, but with the outside observer. You, the observer, could be convinced by the evidence that a Chinese speaker is in the room despite that not being the case. So you can't use outcomes to infer intelligence, and formal computaion of symbols is not thought. A neural network is still just an extremely fancy, well tuned lookup table.

deleted 7y ago

[deleted]

tgv 7y ago

Certainly a lot of exams are filled in using tricks, bordering on superstition. Words that remind a student of a certain kind of solution can cause application of the wrong formula. In that sense, ML, statistical, or "deep learning" models perform a bit like us. Not surprisingly, since co-occurrence plays a large role in our fast intuitions. It also shows that such models are miles away from higher level reasoning.

pmoriarty 7y ago

"At some point we're going to start asking the same questions about humans. Do humans really understand what we read, or have we just picked up weird tricks that happen to work?" That question has been asked by philosophers and psychologists for a long time. There's a branch of philosophy called epistemology -- the study of how we know what we know -- where questions about whether we even know what we claim to know (or understand what we claim to understand) belong. In addition, nearly a hundred years ago behaviorist psychologists were positing that all we can see is other people's behavior, so making claims about their supposed mind states, understanding, etc, were unwarranted and effectively off-limits... and then psychology, cognitive science and linguistics had a backlash against behaviorists, rejecting their approach and assumptions, with mental states, thought, and understanding being admissible in the field for many decades now. So none of this is really new, and mountains of books and papers have been written discussing and debating such issues for at least a hundred years. The Chinese Room thought experiment discussed in the article has been a staple of many Philosophy 101 courses, where the obvious followup question to ask (even for students just getting their first exposure to philosophy) is whether humans understand what they read to begin with. Pick up any related philosophy book and they'll discuss such questions ad nauseum. Taking a much further step back in to history, more than 2000 years ago, the Skeptics were questioning whether we can know anything, and some were denying that anything can be known. Philosophers in the West have been debating such questions ever since. In Eastern philosophy and religion, such concerns are also very old, with a variety of people asserting the illusory nature of worldly knowledge, and positing that true reality is unknowable, or at least not accessible or limited by rational thought. Similar themes are echoed in Western mysticism. Jumping forward yet again (forgive the rambling nature of this response), we have Descartes Duck[1], and the popular Enlightenment and Post-Enlightenment view that "man is just a machine". Questions about whether we really understand or are really doing no more than a sophisticated version of what the Chinese Room or "dumb" neural networks do could be seen as modern restatements of that view. [1] - https://en.wikipedia.org/wiki/Digesting_Duck https://en.wikipedia.org/wiki/Digesting_Duck

klodolph 7y ago
The Chinese room thought experiment is based on logically unsound reasoning and I don’t understand why it has taken hold as widely as it has. The logical argument is that a composite object of a book, room, and person following directions cannot understand Chinese, but there is no justification for this claim—it is, as far as I can tell, a complete non-sequitur. Perhaps I just misunderstand it? Because the logic underlying the Chinese room argument is unsound, it naturally leads to some bizarre conclusions. I think it’s a bit of a tragedy that it’s a staple of philosophy 101 courses, unless you use it as an example for teaching logic by dissecting it and showing how it is unsound. Personally my view is that as much as we need our scientists and ML researchers to study epistemology, we need our epistemologists to study science and mathematics. Any coherent and comprehensive epistemological system in the future will have to somehow synthesize things like substance monism, post-Fisherian / post-Bayesian statistics, post-Popper science, and computational complexity theory. Just as too many ML researchers are reluctant to dip into philosophy and reinvent a poorer version of old philosohpical arguments, there are too many philosophers that would rather e.g. rehash some ideas of Hegel rather than step outside their own field long enough to synthesize new ideas.
- jowday 7y ago
  I never interpreted it as implying the 'composite object of a book, room, and the person following directions' can't understand Chinese, but rather that the person doesn't understand Chinese.
- kthejoker2 7y ago
  Yes, you misunderstand it. The Chinese Room is a refutation of the Turing Test and similar claims on AGI. It has nothing to do with the person in the room, but with the outside observer. You, the observer, could be convinced by the evidence that a Chinese speaker is in the room despite that not being the case. So you can't use outcomes to infer intelligence, and formal computaion of symbols is not thought. A neural network is still just an extremely fancy, well tuned lookup table.