4 ms·

This has been posted many times before, but I thought of it yesterday when I saw this essay by Chomsky in the New York Times, "The False Promise of ChatGPT." h

This has been posted many times before, but I thought of it yesterday when I saw this essay by Chomsky in the New York Times, "The False Promise of ChatGPT." https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chatgpt-ai.html https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chat...

A4ET8a8uTh0 4y ago
He is very smart and obviously an authority in his chosen field, but I am not sure he is right here, because he misunderstands the promise in place ( possibly because he only used it for text generation ). edit: By that I mean the following: "Note, for all the seemingly sophisticated thought and language, the moral indifference born of unintelligence. Here, ChatGPT exhibits something like the banality of evil: plagiarism and apathy and obviation. " He does not seem to understand it is a feature of the system. He is absolutely correct about the level of hype though.
- ericpauley 4y ago
  This was my biggest issue with this article as well. Chomsky (et al.) completely ignores the fact that ChatGPT has been "trained"/gaslit into thinking it is incapable of having an opinion. That ChatGPT returns an Open AI form letter for the questions they ask is almost akin to an exception that proves the rule: ChatGPT is so eager to espouse opinions that OpenAI had to nerf it so it doesn't. Typing the prompts from the article after the DAN (11.0) prompt caused GPT to immediately respond with its opinion. Chomsky's claims in the article are also weak because (as with many discussions about ChatGPT) they are non-falsifiable. There is seemingly no output ChatGPT could produce that would qualify as intelligent for Chomsky. Similar to the Chinese room argument, one can always claim the computer is just emulating understanding.
  - katmannthree 4y ago
    > one can always claim the {entity} is just emulating understanding. I've yet to see a convincing argument that humans are any different. They're sometimes better at pretending to understand things, but at the end of the day both humans and ChatGPT have a small handful of things which they functionally understand[0] and a larger body of knowledge which is only partially integrated. Chomsky has disappeared up his own intestinal tract on this one. One can quibble about intelligence until the end of time, but the real question is that of utility -- which they certainly do have, in ever-increasing scope and measure. [0]: i.e. have synthesized the object and can properly explain and apply it in other contexts
    - A4ET8a8uTh0 4y ago
      I think you have an interesting point. How many of us actually understand how most of our current advancements work? If society as we know it ended now, would I be able to recreate even the basic niceties ( fire, electricity, clean water, stable food supply, ice cars, circuits and so on )? We use all those amazing tools while knowing only a fraction on how they actually work ( or what to do when they break ). Do we merely mimic or do we understand? GPT brought us to an interesting philosophical ledge. edit: somewhat related tangent My extended family member recently claimed she is a conscious consumer unaffected by advertising and therefore not concerned about targeted ads. Is she conscious if she picks what everyone around her picks as a way to fit into society or does she understand her choice, underlying forces and simply opts into them?
  - typon 4y ago
    >Similar to the Chinese room argument, one can always claim the computer is just emulating understanding. And you'd be correct. The point isn't what kind of output ChatGPT can produce, the point is what kind of input it takes to create the model. If ChatGPT were to gain language understanding at the level of a toddler trained on the same number of tokens that a toddler needs, then you could start postulating that the model is really learning, rather than becoming a sophisticated stochastic parrot.
    - A4ET8a8uTh0 4y ago
      I have a small kid now.. and this is exactly what they do: parrot. They parrot more as their source of parrot material expands. ChatGPT is an infant now. More parrot material can be provided.
      - typon 4y ago
        Maybe research done by linguists and cognitive scientists over decades should be valued more than your observation?
        A4ET8a8uTh0 4y ago
        It is a valid point. I am not an authority on linguistics and Chomsky absolutely is ( to the best of my knowledge his theories and contributions to linguistics were not disproved ). However, and this is part my original argument: he is not arguing linguistics. He is arguing intelligence. No. More than that. He is actually arguing an unintelligence, which is a new abstract concept and one that is not a colloquially used. I may be wrong, but if that is the case, and we are still arguing with appeal to authority, shouldn't he defer to experts in that field? Shouldn't AI experts opinion be valued more than his observation? Note, I am merely raising a possibility that he is wrong about this particular idea. More to the point, do you think his understanding of syntax really means he understands the underpinnings of AI; the same AI that merely tries to emulate human language capability?
- aj7 4y ago
  If “plagiarism and apathy and obviation” are a feature, not a bug, then …
  - A4ET8a8uTh0 4y ago
    It is difficult for me to finish this thought. Would you be willing to elaborate?
avgcorrection 4y ago
Not often that Chomsky gets to write something in the NYT, I think. I guess he just has to stay away from topics like East Timor.
walnutclosefarm 4y ago
Chomsky says: The human mind is not, like ChatGPT and its ilk, a lumbering statistical engine for pattern matching, gorging on hundreds of terabytes of data and extrapolating the most likely conversational response or most probable answer to a scientific question. Had he said "not ONLY a statistical engine for pattern matching," I would agree. But I'm pretty sure that the majority of what human cognition is and does is exactly what he describes. The pattern matching engine encodes the wisdom of thousands of years of evolution, in addition to the experiences of any particular instance (person), but in the end, most of what we do and say is generated by predicting, based on pattern matching, what is most likely to get the brain positive feedback. The thing that the rush to more and bigger LLMs seems to leave out, to me, is that the pattern matching and prediction that drives intelligent response and behavior in humans (when you can find it), is not merely prediction about language, but rather prediction based on multiple learned (by humanity generally through evolutionary incorporation into our neural architecture, and by individuals through experience) models - of space, time, biology, abstract reasoning, ontologies that objectify the natural and human intellectual worlds, and more. LLMs encode very little if any of that directly, but rather get pieces of it indirectly through the imprint of ontology and reason that are baked into word (token) usage patterns. So, I agree with Chomsky on the main point: you can't really get much beyond fluency with LLMs alone. There is way too much hype on these things.
- jackmott42 4y ago
  I have a lot of hype because of how capable these systems are with what is essentially a naive and simple approach. (granted with TONS of data thrown at it). Adding in layers of complexity is going to happen next, and I expect it is going to get wild.
  - ChatGTP 4y ago
    We’ll that’s assuming adding in layers of complexity yields fruitful. I can think of a lot of systems that don’t necessarily benefit from being more complex.
- dragonwriter 4y ago
  > Had he said "not ONLY a statistical engine for pattern matching," I would agree. I’m not sure I would even then. I mean maybe, but that’s not the clearest, most certain, fundamental difference. A more clear fundamental difference is ChatGPT instances are all, metaphorically, instinct with no space for intelligence. They have lots of “training”, but that all happens before they are capable of acting, and once they are capable of acting their behavior is entirely preprogrammed based on a very small input window. They have no memory of experience (that’s simulated, within the token limit, between the model and the user for chatbots), much less a reward mechanism that would let them learn behavior from experience.
anybodyz 4y ago
I tried Chomsky's counterfactual apple questions with Chat GPT. It was able to infer counterfactuals about apples or any sort of physical object, both forwards and backwards in time. So I assume Chomsky has not spent much time with ChatGPT or he would know that it was capable of making physical inferences and counterfactual inferences like that. Not perfectly (it did make a few mistakes) but fairly reliably. That ChatGPT regularly "makes stuff up" and that there is no difference between the truth or falsehood of any statement also seems to be false. I asked ChatGPT to act as a "[Lie Detector]" and to rate the truth or falsehood of a variety of statements asked. I asked about 40 questions ranging from physical situations (heavy objects floating away into air) temporal questions (time travel, etc.) and logical questions - and it very accurately could determine if each of these statements was "true" or "false" given physical or logical rules. Again not perfect but very accurate (38 out of 40 correct). With attention - ChatGPT is very obviously operating at a level above the simple probabilistic prediction. It clearly seems that it has some notion as to the meaning of what is being said and is making inferences based on that meaning. That those inferences were trained probabilistically is certainly true, but that it was trained on the average human's understanding of those physical or temporal or other constraints also seems to be true and to also be fairly accurate.
- dwallin 4y ago
  I think a lot of the novel uses of language models like chatgpt will involve multiple instances interacting upon interleaved data. For example to improve factuality you might do the following: 1) One instance first parses the chat and last message to generate a response. Currently this is where things end but we can keep this private and do additional work. 2) A second instance, properly primed, can take the last prompt and response and "analyze" it, generating scores for things like factuality and usefulness, possibly adding commentary. 3) Pass into a third instance that has the chat history again to rewrite the response, taking into account the feedback. 4) Optionally repeat #2 and #3 until it passes some quality threshold.

A4ET8a8uTh0 4y ago

He is very smart and obviously an authority in his chosen field, but I am not sure he is right here, because he misunderstands the promise in place ( possibly because he only used it for text generation ). edit: By that I mean the following: "Note, for all the seemingly sophisticated thought and language, the moral indifference born of unintelligence. Here, ChatGPT exhibits something like the banality of evil: plagiarism and apathy and obviation. " He does not seem to understand it is a feature of the system. He is absolutely correct about the level of hype though.

ericpauley 4y ago
This was my biggest issue with this article as well. Chomsky (et al.) completely ignores the fact that ChatGPT has been "trained"/gaslit into thinking it is incapable of having an opinion. That ChatGPT returns an Open AI form letter for the questions they ask is almost akin to an exception that proves the rule: ChatGPT is so eager to espouse opinions that OpenAI had to nerf it so it doesn't. Typing the prompts from the article after the DAN (11.0) prompt caused GPT to immediately respond with its opinion. Chomsky's claims in the article are also weak because (as with many discussions about ChatGPT) they are non-falsifiable. There is seemingly no output ChatGPT could produce that would qualify as intelligent for Chomsky. Similar to the Chinese room argument, one can always claim the computer is just emulating understanding.
- katmannthree 4y ago
  > one can always claim the {entity} is just emulating understanding. I've yet to see a convincing argument that humans are any different. They're sometimes better at pretending to understand things, but at the end of the day both humans and ChatGPT have a small handful of things which they functionally understand[0] and a larger body of knowledge which is only partially integrated. Chomsky has disappeared up his own intestinal tract on this one. One can quibble about intelligence until the end of time, but the real question is that of utility -- which they certainly do have, in ever-increasing scope and measure. [0]: i.e. have synthesized the object and can properly explain and apply it in other contexts
  - A4ET8a8uTh0 4y ago
    I think you have an interesting point. How many of us actually understand how most of our current advancements work? If society as we know it ended now, would I be able to recreate even the basic niceties ( fire, electricity, clean water, stable food supply, ice cars, circuits and so on )? We use all those amazing tools while knowing only a fraction on how they actually work ( or what to do when they break ). Do we merely mimic or do we understand? GPT brought us to an interesting philosophical ledge. edit: somewhat related tangent My extended family member recently claimed she is a conscious consumer unaffected by advertising and therefore not concerned about targeted ads. Is she conscious if she picks what everyone around her picks as a way to fit into society or does she understand her choice, underlying forces and simply opts into them?
- typon 4y ago
  >Similar to the Chinese room argument, one can always claim the computer is just emulating understanding. And you'd be correct. The point isn't what kind of output ChatGPT can produce, the point is what kind of input it takes to create the model. If ChatGPT were to gain language understanding at the level of a toddler trained on the same number of tokens that a toddler needs, then you could start postulating that the model is really learning, rather than becoming a sophisticated stochastic parrot.
  - A4ET8a8uTh0 4y ago
    I have a small kid now.. and this is exactly what they do: parrot. They parrot more as their source of parrot material expands. ChatGPT is an infant now. More parrot material can be provided.
    - typon 4y ago
      Maybe research done by linguists and cognitive scientists over decades should be valued more than your observation?
      - A4ET8a8uTh0 4y ago
        It is a valid point. I am not an authority on linguistics and Chomsky absolutely is ( to the best of my knowledge his theories and contributions to linguistics were not disproved ). However, and this is part my original argument: he is not arguing linguistics. He is arguing intelligence. No. More than that. He is actually arguing an unintelligence, which is a new abstract concept and one that is not a colloquially used. I may be wrong, but if that is the case, and we are still arguing with appeal to authority, shouldn't he defer to experts in that field? Shouldn't AI experts opinion be valued more than his observation? Note, I am merely raising a possibility that he is wrong about this particular idea. More to the point, do you think his understanding of syntax really means he understands the underpinnings of AI; the same AI that merely tries to emulate human language capability?
aj7 4y ago
If “plagiarism and apathy and obviation” are a feature, not a bug, then …
- A4ET8a8uTh0 4y ago
  It is difficult for me to finish this thought. Would you be willing to elaborate?

avgcorrection 4y ago

Not often that Chomsky gets to write something in the NYT, I think. I guess he just has to stay away from topics like East Timor.

walnutclosefarm 4y ago

Chomsky says: The human mind is not, like ChatGPT and its ilk, a lumbering statistical engine for pattern matching, gorging on hundreds of terabytes of data and extrapolating the most likely conversational response or most probable answer to a scientific question. Had he said "not ONLY a statistical engine for pattern matching," I would agree. But I'm pretty sure that the majority of what human cognition is and does is exactly what he describes. The pattern matching engine encodes the wisdom of thousands of years of evolution, in addition to the experiences of any particular instance (person), but in the end, most of what we do and say is generated by predicting, based on pattern matching, what is most likely to get the brain positive feedback. The thing that the rush to more and bigger LLMs seems to leave out, to me, is that the pattern matching and prediction that drives intelligent response and behavior in humans (when you can find it), is not merely prediction about language, but rather prediction based on multiple learned (by humanity generally through evolutionary incorporation into our neural architecture, and by individuals through experience) models - of space, time, biology, abstract reasoning, ontologies that objectify the natural and human intellectual worlds, and more. LLMs encode very little if any of that directly, but rather get pieces of it indirectly through the imprint of ontology and reason that are baked into word (token) usage patterns. So, I agree with Chomsky on the main point: you can't really get much beyond fluency with LLMs alone. There is way too much hype on these things.

jackmott42 4y ago
I have a lot of hype because of how capable these systems are with what is essentially a naive and simple approach. (granted with TONS of data thrown at it). Adding in layers of complexity is going to happen next, and I expect it is going to get wild.
- ChatGTP 4y ago
  We’ll that’s assuming adding in layers of complexity yields fruitful. I can think of a lot of systems that don’t necessarily benefit from being more complex.
dragonwriter 4y ago
> Had he said "not ONLY a statistical engine for pattern matching," I would agree. I’m not sure I would even then. I mean maybe, but that’s not the clearest, most certain, fundamental difference. A more clear fundamental difference is ChatGPT instances are all, metaphorically, instinct with no space for intelligence. They have lots of “training”, but that all happens before they are capable of acting, and once they are capable of acting their behavior is entirely preprogrammed based on a very small input window. They have no memory of experience (that’s simulated, within the token limit, between the model and the user for chatbots), much less a reward mechanism that would let them learn behavior from experience.

anybodyz 4y ago

I tried Chomsky's counterfactual apple questions with Chat GPT. It was able to infer counterfactuals about apples or any sort of physical object, both forwards and backwards in time. So I assume Chomsky has not spent much time with ChatGPT or he would know that it was capable of making physical inferences and counterfactual inferences like that. Not perfectly (it did make a few mistakes) but fairly reliably. That ChatGPT regularly "makes stuff up" and that there is no difference between the truth or falsehood of any statement also seems to be false. I asked ChatGPT to act as a "[Lie Detector]" and to rate the truth or falsehood of a variety of statements asked. I asked about 40 questions ranging from physical situations (heavy objects floating away into air) temporal questions (time travel, etc.) and logical questions - and it very accurately could determine if each of these statements was "true" or "false" given physical or logical rules. Again not perfect but very accurate (38 out of 40 correct). With attention - ChatGPT is very obviously operating at a level above the simple probabilistic prediction. It clearly seems that it has some notion as to the meaning of what is being said and is making inferences based on that meaning. That those inferences were trained probabilistically is certainly true, but that it was trained on the average human's understanding of those physical or temporal or other constraints also seems to be true and to also be fairly accurate.

dwallin 4y ago
I think a lot of the novel uses of language models like chatgpt will involve multiple instances interacting upon interleaved data. For example to improve factuality you might do the following: 1) One instance first parses the chat and last message to generate a response. Currently this is where things end but we can keep this private and do additional work. 2) A second instance, properly primed, can take the last prompt and response and "analyze" it, generating scores for things like factuality and usefulness, possibly adding commentary. 3) Pass into a third instance that has the chat history again to rewrite the response, taking into account the feedback. 4) Optionally repeat #2 and #3 until it passes some quality threshold.