3 ms·

I was in the audience during the panel and Chomsky was really trying to hold onto a view of science disconnected from the engineering needed to replicate the na

by robg 4y ago

I was in the audience during the panel and Chomsky was really trying to hold onto a view of science disconnected from the engineering needed to replicate the natural phenomenon. To take a simple example: Suppose you’re the Wright Brothers and you built a flying machine. Chomsky’s response would be: But you didn’t explain how birds fly. On the one hand, sure. Birds fly differently. On the other, the machine flies. Why does it need to explain all of flight? The same is true of language models. They don’t explain how we acquire language. But they replicate language uses in ways that look like language speakers.

paganel 4y ago
It’s a valid criticism. What today’s “language” models do is that they have been able to scale up the implementation of the Mechanical Turk problem. They have done that quite well, give or take a few errors. The issue is that there’s a whole another world outside of the “answers” provided by Mechanicak Turk-like solutions, this is what guys like Chomsky allude to.
BoardsOfCanada 4y ago
I think that part of Chomsky's argument would be that language primarily is a tool for thinking and reasoning, and that the externalization and use for communication came later (I remember this from a lecture of his). In that view "language" of an AI is not the same as human language, while flight is flight regardless if it's mechanical or biological.
- robg 4y ago
  Jerry Fodor used to write of mentalese. It’s a fair criticism. And yet generative AI is showing that the models can be creative in ways that look like thinking outside of language.
foobarqux 4y ago
Chomsky has explicitly made the distinction between the difference between a useful tool and science, which tries to explain things. In fact he does it almost every time he talks about this subject.
m_nyongesa 4y ago
The major difference between the airplane flying example and the computer language reproduction model example is that we actually understand how aerodynamics and flight work, so we understand very well how an airplane flies (it doesn't just "replicate" flight). We don't understand how the human brain works so that our minds can acquire language.
- wesleychen 4y ago
  Explanations of how airplanes fly are remarkably similar to explanations of how LNNs work: https://www.wired.com/story/theres-no-one-way-to-explain-how-flying-works/ https://www.wired.com/story/theres-no-one-way-to-explain-how... You have probably heard of the explanation that the curvature of the airplane's wings generates lift. However, this is wrong (as explained in the article) and can't explain observable phenomena such as planes flying upside down.

by robg 4y ago

paganel 4y ago

It’s a valid criticism. What today’s “language” models do is that they have been able to scale up the implementation of the Mechanical Turk problem. They have done that quite well, give or take a few errors. The issue is that there’s a whole another world outside of the “answers” provided by Mechanicak Turk-like solutions, this is what guys like Chomsky allude to.

BoardsOfCanada 4y ago

I think that part of Chomsky's argument would be that language primarily is a tool for thinking and reasoning, and that the externalization and use for communication came later (I remember this from a lecture of his). In that view "language" of an AI is not the same as human language, while flight is flight regardless if it's mechanical or biological.

robg 4y ago
Jerry Fodor used to write of mentalese. It’s a fair criticism. And yet generative AI is showing that the models can be creative in ways that look like thinking outside of language.

foobarqux 4y ago

Chomsky has explicitly made the distinction between the difference between a useful tool and science, which tries to explain things. In fact he does it almost every time he talks about this subject.

m_nyongesa 4y ago

The major difference between the airplane flying example and the computer language reproduction model example is that we actually understand how aerodynamics and flight work, so we understand very well how an airplane flies (it doesn't just "replicate" flight). We don't understand how the human brain works so that our minds can acquire language.

wesleychen 4y ago
Explanations of how airplanes fly are remarkably similar to explanations of how LNNs work: https://www.wired.com/story/theres-no-one-way-to-explain-how-flying-works/ https://www.wired.com/story/theres-no-one-way-to-explain-how... You have probably heard of the explanation that the curvature of the airplane's wings generates lift. However, this is wrong (as explained in the article) and can't explain observable phenomena such as planes flying upside down.