5 ms·

"In this piece Chomsky, the intellectual godfather of an effort that failed for 60 years to build machines that can converse in ordinary language, condemns the

"In this piece Chomsky, the intellectual godfather of an effort that failed for 60 years to build machines that can converse in ordinary language, condemns the effort that succeeded." That's incorrect on a few levels. Aaronson is talking about something else, different from what Chomsky worked on. ChatGPT can put together a statistically likely series of tokens, but ChatGPT doesn't understand the meaning of those tokens, and therefore ChatGPT has no concept of "truth." ChatGPT cannot deliberately lie or tell the truth, it has no understanding of such things. By contrast, Chomsky, for much of his career, was a linguist and he focused on the issues of how is knowledge constructed, and how do we know what truth is, and how does language express this? So Chomsky is pointing out that ChatGPT creates a fascinating illusion of a real conversation, but it isn't exactly what AI researchers were aiming for, for several decades. And it is appropriate that the New York Times would want to publish an essay that speaks clearly about what ChatGPT is and is not, because otherwise there is a risk that the general public will get an over-hyped view of what ChatGPT does.

gnramires 4y ago
> therefore ChatGPT has no concept of "truth." ChatGPT cannot deliberately lie or tell the truth, it has no understanding of such things Please see my comment below[1], this is not a fair characterization. ChatGPT is not a pure predictive LLM, it also undergoes RLHF which gives it a much more human-like character, including clearly having a concept of truth within its limits. [1] https://news.ycombinator.com/item?id=35080933 https://news.ycombinator.com/item?id=35080933
- tpoacher 4y ago
  Indeed. I would say a better criticism wouldn't be that it has no notion of truth, but that it has no ability to reason on the basis of a reinterpretation of what might or should constitute truth. Even if you come up with a fancy prompt that asks it to "assume" certain things, it will still have to base that assumption on things that it already 'knows' to be true, as it already perceives them to be true.
deleted 4y ago
[deleted]
deleted 4y ago
[deleted]
dekhn 4y ago
How sure are you that humans understand the meaning of tokens, have concept of truth, or understanding? Ignore your subjective experience of consciousness and ask, "what scientific experiment could I run that would distinguish the special nature of humans from sufficiently magical AI?"
epilys 4y ago
I agree. It seems to lack self awareness, considering how some weeks ago there was a post on his blog about experts in unrelated fields who claimed a wormhole simulation was the creation of an actual wormhole. Aaronson said that believing these claims is an example of Gell-Mann amnesia. Why does Aaronson think he's suddenly an expert in AI, language and consciousness? Being correct in quantum computing topics doesn't translate to this field at all. And worse than that, there are no real arguments here, just misrepresentation of Chomsky and ad hominems.
- kgwgk 4y ago
  Being an expert in quantum computing doesn’t even make him an expert in physics.
pharrington 4y ago
> ChatGPT doesn't understand the meaning of those tokens, and therefore ChatGPT has no concept of "truth." ChatGPT cannot deliberately lie or tell the truth, it has no understanding of such things It does encode meaning, but just knowing things doesn't give you the ability to know if they are true. A human brain in a jar that was only fed text from the internet also wouldn't be able to verify what it learned against reality. Chomsky has the flawed conception of language. It is just one tool (though a very powerful one) that humans use when reasoning.
- YeGoblynQueenne 4y ago
  Language models don't encode meaning. They "encode" the probability of collocations between lexical tokens. Such collocations are correlated with meaning, but they do not "encode" meaning. We don't know how collocation of tokens is related with meaning, we don't understand why certain words, in a certain order, mean a particular thing and not another, and so we don't know how to model that relation. For example, suppose I give you a dictionary of a language you don't know. Could you learn the meaning of words in the language, by reading their lemmas in the dictionary? Note that words in a dictionary are collocated with explanations of them, not just other words, or tokens. I'm Greek, so suppose you didn't speak Greek and I gave you a dictionary of Greek, where you could find the following: Αβαρκαμαξας: ιμιά καρός μαύρα φούμαρα σαχλαπαράκατα μαρακαφούλες. Can you tell what the word "Αβαρκαμαξας" means, just by looking at the dictionary entry above? Obviously you can't (and you can't translate it in Google Translate either because it, and its explanation, are completely made up and not even real Greek). In the same way, just matching tokens with each other based on where they appear next to each other in text doesn't say anything about their meaning. We don't know how to encode meaning. Statistical language modelling is exactly an attempt to represent language, without having to tackle the problem of representing meaning.
  - EnergyAmy 4y ago
    And yet, what if you had an entire internet of ungreek like that and spent the equivalent of many human lifetimes reading through it? You would encode your own meaning, just like a large language model does internally. Your encoded meaning might not be quite right in the end, but as the saying goes, all models are wrong, but some are useful.
testrun 4y ago
Watching commentators from MSNBC to Fox, from pro-Russian to pro-Ukrainian, from pro-Israel to anti-Israel and so on and on and on, I don't think we humans have a good understanding of truth either. In fact I read Chomsky's comments in regards to Serbia and Russia, and I think truth is in the eye of the beholder.

gnramires 4y ago

> therefore ChatGPT has no concept of "truth." ChatGPT cannot deliberately lie or tell the truth, it has no understanding of such things Please see my comment below[1], this is not a fair characterization. ChatGPT is not a pure predictive LLM, it also undergoes RLHF which gives it a much more human-like character, including clearly having a concept of truth within its limits. [1] https://news.ycombinator.com/item?id=35080933 https://news.ycombinator.com/item?id=35080933

tpoacher 4y ago
Indeed. I would say a better criticism wouldn't be that it has no notion of truth, but that it has no ability to reason on the basis of a reinterpretation of what might or should constitute truth. Even if you come up with a fancy prompt that asks it to "assume" certain things, it will still have to base that assumption on things that it already 'knows' to be true, as it already perceives them to be true.

deleted 4y ago

[deleted]

deleted 4y ago

[deleted]

dekhn 4y ago

How sure are you that humans understand the meaning of tokens, have concept of truth, or understanding? Ignore your subjective experience of consciousness and ask, "what scientific experiment could I run that would distinguish the special nature of humans from sufficiently magical AI?"

epilys 4y ago

I agree. It seems to lack self awareness, considering how some weeks ago there was a post on his blog about experts in unrelated fields who claimed a wormhole simulation was the creation of an actual wormhole. Aaronson said that believing these claims is an example of Gell-Mann amnesia. Why does Aaronson think he's suddenly an expert in AI, language and consciousness? Being correct in quantum computing topics doesn't translate to this field at all. And worse than that, there are no real arguments here, just misrepresentation of Chomsky and ad hominems.

kgwgk 4y ago
Being an expert in quantum computing doesn’t even make him an expert in physics.

pharrington 4y ago

> ChatGPT doesn't understand the meaning of those tokens, and therefore ChatGPT has no concept of "truth." ChatGPT cannot deliberately lie or tell the truth, it has no understanding of such things It does encode meaning, but just knowing things doesn't give you the ability to know if they are true. A human brain in a jar that was only fed text from the internet also wouldn't be able to verify what it learned against reality. Chomsky has the flawed conception of language. It is just one tool (though a very powerful one) that humans use when reasoning.

YeGoblynQueenne 4y ago
Language models don't encode meaning. They "encode" the probability of collocations between lexical tokens. Such collocations are correlated with meaning, but they do not "encode" meaning. We don't know how collocation of tokens is related with meaning, we don't understand why certain words, in a certain order, mean a particular thing and not another, and so we don't know how to model that relation. For example, suppose I give you a dictionary of a language you don't know. Could you learn the meaning of words in the language, by reading their lemmas in the dictionary? Note that words in a dictionary are collocated with explanations of them, not just other words, or tokens. I'm Greek, so suppose you didn't speak Greek and I gave you a dictionary of Greek, where you could find the following: Αβαρκαμαξας: ιμιά καρός μαύρα φούμαρα σαχλαπαράκατα μαρακαφούλες. Can you tell what the word "Αβαρκαμαξας" means, just by looking at the dictionary entry above? Obviously you can't (and you can't translate it in Google Translate either because it, and its explanation, are completely made up and not even real Greek). In the same way, just matching tokens with each other based on where they appear next to each other in text doesn't say anything about their meaning. We don't know how to encode meaning. Statistical language modelling is exactly an attempt to represent language, without having to tackle the problem of representing meaning.
- EnergyAmy 4y ago
  And yet, what if you had an entire internet of ungreek like that and spent the equivalent of many human lifetimes reading through it? You would encode your own meaning, just like a large language model does internally. Your encoded meaning might not be quite right in the end, but as the saying goes, all models are wrong, but some are useful.

testrun 4y ago

Watching commentators from MSNBC to Fox, from pro-Russian to pro-Ukrainian, from pro-Israel to anti-Israel and so on and on and on, I don't think we humans have a good understanding of truth either. In fact I read Chomsky's comments in regards to Serbia and Russia, and I think truth is in the eye of the beholder.