8 ms·

Caricaturing Noam Chomsky

58 points by bundie 4y ago

t344344 4y ago
Just a bit of context (not related to AI). Chomsky appeared on pro Russian podcasts and expressed some controversial opinions.
- unconed 4y ago
  Is this a satirical comment about how people try to inject blatant guilt by association under the guise of "context"?
  - deleted 4y ago
    [deleted]
  - t344344 4y ago
    No, warning readers that Chomsky may be under usual smear campaign.
  - hackandthink 4y ago
    A nice reddit thread about some Chomsky critics: https://www.reddit.com/r/chomsky/comments/11gfo8m/why_are_chomsky_haters_so_aggressive/ https://www.reddit.com/r/chomsky/comments/11gfo8m/why_are_ch...
z3c0 4y ago
I too agree with Chomsky. It's concerning to see people piling onto GPT/LLMs as an extractive tool - something it does okay at, but in the end, something it's not made for. So many companies rushing to use it as such is going to result in some colossal problems for some. The potential for completely false assertions is not something one should be in a hurry to productionize. There's already a myriad of extractive NLP solutions available that organizations could be utilizing (like any simpler BERT-based model.) Unfortunately, they don't come with the veneer of misleading ethos that GPT/LLMs pretend to have. Personally, I see some potential for customer-facing uses, but that's because customers like novelties.
- bundie 4y ago
  Do you think AGI will be achieved if this trend continues?
  - z3c0 4y ago
    Not really, no, but that's not to say that LLMs couldn't be a useful shell through which an AGI could interact with the rest of the world. But to herald LLMs as the coming of AGI would be to put the cart before the horse. A rudimentary intelligence capable of basic object-recognition and permanence* would be an actual step towards AGI over LLMs, in my professional (but still fallible) opinion. Something as basic as an insect, or hell, even protozoa, is all we would need to say that we're on the way. Once we have that, tying it with an LLM to give it denotative capabilities would be the boom we're all hoping for/terrified of. [*] this is key, but we don't need to codify this so much as find a way for the object-recognition to build up to it. Even children take a while to develop object permanence (hence peek-a-boo being fun for them - you really are disappearing.)
    - dr_dshiv 4y ago
      I really don’t understand why it isn’t plausible that LLMs are weak AGI. If we’d been wise enough to create benchmarks for them a decade ago, I’m sure they would have exceed our expectations. The generality of the intelligence is unquestionable, at least in a language domain. Discounting it as AGI because a fly is better at being a fly seems besides the point.
      - goatlover 4y ago
        Because they don't do anything but generate text when prompted, while humans are walking around doing things. I would expect at least two things from an AGI, weak or strong: 1. It has agency. 2. It can move about and manipulate the world in a general as opposed to predefined sense.
        Dzugaru 4y ago
ev7 4y ago
Enjoyed the content of the article, but its credibility is diminished a bit by its numerous typos.
- mtlmtlmtlmtl 4y ago
  Uhm, how?
hackandthink 4y ago
Non paywall link to Chomsky's essay: https://dnyuz.com/2023/03/08/noam-chomsky-the-false-promise-of-chatgpt/ https://dnyuz.com/2023/03/08/noam-chomsky-the-false-promise-...
- greenyoda 4y ago
  And the HN discussion of it: https://news.ycombinator.com/item?id=35067619 https://news.ycombinator.com/item?id=35067619
bmitc 4y ago
An observation of mine with regards to ardent Chomsky critics is that it's usually the case that the critic has a lot to gain from their attack or position while Chomsky has little to nothing to gain. I think the whole stable diffusion and ChatGPT thing will come and go, almost as self-driving cars have. If you remember, just eight or so years ago, everybody was absolutely convinced that they were right around the corner. There was an insane amount of hype. Now, that hype has largely come and gone, with several companies abandoning their self-driving approaches or just flat out failing at them. There is still some hype, but I feel it has faded as people have come to grips with the premise, much less the difficulty of the implementations. The focus seems to be shifting back to driver assistance features. Another thing is VR. So these stable diffusion and GPT methods are having their time in the sun. Yet again, just like VR and self-driving, proponents are trying to will these things into existence. Yes, they are impressive in some very specific cases, but they are not general. Just like stable diffusion, these GPT things will just flood the Internet with junk, either filling people with incorrect knowledge or driving people away, leaving the only remaining application to be advertising and trying to sell stuff. Humans are seemingly absolutely obsessed with technology trying to replace humans and reality. That is the common thread between all of VR, self-driving, stable diffusion, and now ChatGPT. But this is both hard and destined for failure. People keep discovering that the role of technology is to augment humans and not to replace them. Engelbart knew this 60 years ago.
- mishkovski 4y ago
  That is why Apple’s next big product will probably be an application of AR.
- goatlover 4y ago
  Don't forget the 3D/additive printing revolution that was going to put a printer in everyone's home so they wouldn't need to buy stuff from Walmart anymore. Everyone knows the hype cycle is real, and yet everybody still gets caught up during the early hype.
  - code_duck 4y ago
    To me it feels like the late 70s/early 80s and home computing software. These technologies do exist, as far as self driving cars, chatbots that answer all of your questions insightfully and correctly, and 3D printers that can manufacture household goods in demand. There are working examples in use. They’re not quite ready for the mass market, but they will be someday. I’d give it 15-20 years.
331c8c71 4y ago
It's not that I agree with everything Chomsky says or writes but I can't shake the feeling that he's much more intelligent, eloquent and gentlemanly than any of his critics.
- the-grump 4y ago
  +1 I have my own reservations about his opinion but would never state them in the combative and sneering tone often employed by his critics. Can't help but feel like they personally have a problem with him, and they're using this piece as a cudgel to "take a swing".
- tgv 4y ago
  Even though I largely disagree with Chomsky, he is extremely intelligent. That Hinton and Sejnowski behave like sore losers is a bit of a disappointment. The argument with the learning disabled student is a disgrace. It's almost as if they cannot imagine that the human brain has innate properties beyond "weight adaptation".
  - epups 4y ago
    I did not see a problem with Hinton's argument here - perhaps you would care to expand? If we are evaluating LLM's purely on outcome alone, it seems only fair to compare them to humans that don't have fully functioning cognitive capabilities. We wouldn't discredit a schizophrenic person that hallucinates, we know that this person can also see reality for what it is, most of the time. I just don't think that finding absurd examples, usually after careful prompting, is a slam dunk against LLM's at all.
  - naasking 4y ago
    > The argument with the learning disabled student is a disgrace. It's almost as if they cannot imagine that the human brain has innate properties beyond "weight adaptation". They can, it's just that Chomsky's dismissal of "weight adaptations" as insufficient is not actually justified by any evidence. The Transformer model augmented with memory is Turing complete, so it can in principle learn to compute anything using only weight adaptations.
masswerk 4y ago
> To his credit Sejnowski correctly picked on a weak point in the oped: Chomsky’s ChatGPT examples falling apples and gravity were anecdotal and insufficiently nuanced. This, I fail to understand. Clearly, Chomsky started with an exposition of the concept of negativity. How can this be missed by anyone with cursory familiarity with western philosophy? The rest of the article (including the passage regarding the apple) is pretty much a series of excursions on this theme, how LLMs systematically miss negativity.
- masswerk 4y ago
  It may be observed that Chomskys piece is in this sense much a modern take on Hegel's Phenomenology of Spirit, who also starts in his exposition with the core theme of negativity, as the very base of "sense certainty" (better: sensual certainty, sinnliche Gewissheit), what Hegel called the "teachings of Ceres", the very origin of conceptuality, in order to expand on this in his grand scheme. (I find it quite remarkable that Chomsky would fall back on this classic line of thought on this occasion.)
- epups 4y ago
  It's pretty bad because Chomsky provided only two examples of things ChatGPT cannot do - even implying its underlying architecture could not ever do that -, and it's obvious that neither he, his coauthors or the editors at the NYT bothered to fact check these assertions. I would bet that Chomsky himself has never played around with ChatGPT itself.
  - masswerk 4y ago
    If we subscribe for a moment to the tradition that conceptuality, creativity, even certainty, are born from negativity, that crucial "what if not" assumption, the ponderability of existential alternatives, we may indeed defer that this is not something LLMs (in their current configuration) can do. (In a more statistical tradition, proximity to the semantic center is pretty much opposed to this.) Which poses yet another question, namely, how essential and significant are these things for us? Is the alleged promise aligned to our basic assumptions?
roenxi 4y ago
Chomsky may be right or wrong. The problem the philosophers face is the high likelihood that the "a lumbering statistical engine for pattern matching, gorging on hundreds of terabytes of data and extrapolating" [0] is going to just be more effective than whatever it is human brains do, similar to how rocks turn out to be more effective at adding numbers together than a mathematician despite the early indications from prehistory to whenever the calculator became viable. Statistical models turn out to be enough to get past the Turing test. They're probably going to be better at interpreting vague requirements and making decisions than most humans too. They're better at art than the average, and better at explaining themselves than most humans. [0] https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chatgpt-ai.html https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chat...
- apwheele 4y ago
  Reminds me a bit of this blog post by Tal Yarkoni, https://www.talyarkoni.org/blog/2018/08/18/if-we-already-understood-the-brain-would-we-even-know-it/ https://www.talyarkoni.org/blog/2018/08/18/if-we-already-und.... That is perhaps more consistent with the "we are just really complicated neural networks argument".
- mtlmtlmtlmtl 4y ago
  People throw around this claim about the turing test. I've yet to see any empirical corroboration of it. Using an LLM and being amazed by its response is not doing the Turing test. The Turing test is mostly a thought experiment anyway.
  - em-bee 4y ago
    if a machine can pass the turing test then that most likely means that the test was not good enough. it is not an indication that the machine is actually intelligent.
    - mtlmtlmtlmtl 4y ago
      I think ChatGPT would fail the test as described by Turing almost immediately, because at some point it would just tell the other test subject it's a language model. In some sense ChatGPT is actually engineered to fail the test.
azubinski 4y ago
Chomsky is saying things that are completely obvious, and that's where the comedy comes in. The tragedy seems to be that during the hype of any technological bubble, someone of the scale of Chomsky is needed to make at least someone hear the obvious. But maybe it's also a comedy. However, it doesn't matter. All flying cars will be on RISK V and Rust and on the blockchain soon. That's what's main.
- naasking 4y ago
  > Chomsky is saying things that are completely obvious, and that's where the comedy comes in. No, the comedy is that he's saying things that seem completely obvious but are actually completely wrong.
aborsy 4y ago
Aaronson works on quantum complexity theory. He has almost no papers on language or AI (classical one, the one that has produced LLMs in question). He is going to go to OpenAI soon for some sort of position (perhaps PR around quantum ML, that is nevertheless important for the company). Chomsky has around half a million citations to his scientific work, mostly on language. The guy is amazing. At age 94, he is productive, sharp and very well spoken. I tend to take him more seriously on this. One problem is that, experts in one domain opine about domains they have not worked on. Another problem is, some people apparently hold a view that, if you make a product that works and sells, nothing else matters. Everyone will rush to earn a living around it. They feel empowered to take any position. So Science doesn’t matter if it’s not aligned, products do. Aaronson didn’t publish a proper article addressing Chomsky’s arguments.
- mudita 4y ago
  As far as I know, Scott Aaronson has been on leave from UT Austin to work on theoretical foundations of AI safety at OpenAI since June 2022: https://scottaaronson.blog/?p=6484 https://scottaaronson.blog/?p=6484
  - the-grump 4y ago
    That sounds like a conflict of interest. I haven't read his article but would expect this to be noted at the top.
    - mudita 4y ago
      There is a note on Aaronson’s blog post: "[Note for people who might be visiting this blog for the first time: I’m a CS professor at UT Austin, on leave for one year to work at OpenAI on the theoretical foundations of AI safety. I accepted OpenAI’s offer in part because I already held the views here, or something close to them; and given that I could see how large language models were poised to change the world for good and ill, I wanted to be part of the effort to help prevent their misuse. No one at OpenAI asked me to write this or saw it beforehand, and I don’t even know to what extent they agree with it.]"
erwald 4y ago
IMO Chomsky (and Marcus) are mostly wrong on LLMs. Quoting the summary of a post[1] I wrote recently: - LLMs are sometimes said to be “just” shallow pattern matchers, “just” massive look-up tables or “just” autocomplete engines. These comparisons amount to a form of (methodological) reductionism. While there’s some truth to them, I think they smuggle in corollaries that are either false or at least not obviously true. - For example, they seem to imply that what LLMs are doing amounts merely to rote memorisation and/or clever parlour tricks, and that they cannot generalise to out-of-distribution data. In fact, there’s empirical evidence that suggests that LLMs can learn general algorithms and can contain and use representations of the world similar to those we use. - They also seem to suggest that LLMs merely optimise for success on next-token prediction. It’s true that LLMs are (mostly) trained on next-token prediction, and it’s true that this profoundly shapes their output, but we don’t know whether this is how they actually function. We also don’t know what sorts of advanced capabilities can or cannot arise when you train on next-token prediction. - So there’s reason to be cautious when thinking about LLMs. In particular, I think, caution should be exercised (1) when making predictions about what LLMs will or will not in future be capable of and (2) when assuming that such-and-such a thing must or cannot possibly happen inside an LLM. https://www.erichgrunewald.com/posts/against-llm-reductionism/ https://www.erichgrunewald.com/posts/against-llm-reductionis...
- resters 4y ago
  Think of it this way: They create plausible patterns of language that seem (to humans) similar to human-generated language. That is what they are "good at". Imagine an "AI" that "knows" about rhyme and meter and can generate a seemingly infinite collection of catchy songs that follow the II V I chord progression, only the sequence of words doesn't necessarily make any sense. LLMs are a more powerful version of that kind of thing that does put the words in a sequence that feels like natural human language. Since they are trained on information, the sequences chosen often seem informational, even though they are just patterns. It's really a lot like autocomplete. In other words, a lot of useful information may accidentally be encoded into the model, but so is a lot of stuff that looks/feels similar but is nonsense.
  - 4y ago
erwald 4y ago
> Aaronson’s biggest error, now corrected, sort of, in bold, is in assuming that Chomsky has spent his life in some sort of failed effort to build AI, which kind of entirely misses the point of Chomsky’s piece (which says in so many words that we need to study the mind first before we try to make AI) and also utterly misrepresents Chomsky’s career. Frankly, I would be embarrassed to have to publish a correction (the part in bold) like this [...] It’s a wild swing and a miss. Chomsky has spent his career trying to understand how humans acquire language, not “building machines” to try to do the same. This is pretty unfair IMO. Aaronson wrote (emphasis mine): > In this piece Chomsky, THE INTELLECTUAL GODFATHER GOD OF an effort that failed for 60 years to build machines that can converse in ordinary language, condemns the effort that succeeded. He never wrote that Chomsky himself was involved in this efforts, only that he was influential to it (which may or may not be true, but Marcus never argues that point).
epups 4y ago
> If a broken clock were correct twice a day, would we give it credit for patches of understanding of time? If a clock was correct 95% of the time, would you say it has no understanding of time? To me it seems that both Marcus and Chomsky are resorting to a sort of dualistic theory here. They concede that these LLM's are capable of impressive displays of cognition, but wait! It's not real cognition of course, it's fake, unlike the one in humans. Well, we don't know enough about cognition in humans to reject the hypothesis that we are also communicating or thinking based on similar principles. ChatGPT is by far the most successful automaton for understanding and producing language we have ever seen. The language Chomsky uses in the op-ed and in his reply here - "(...)Therefore they are telling us nothing about language, cognition, acquisition, anything" - is pure hubris and completely unjustified.
- resters 4y ago
  When you say that the output of LLM is "correct", what do you mean? The output that seems impressive is merely "plausibly written by an intelligent entity" but there is nothing about it that could be characterized as being "correct". Similarly, we could sprinkle formic acid on the floor in a pattern that fooled ants into following it in search of food, but there is nothing "correct" about it, merely "realistic". Asking GPT3 to create a persuasive essay is impressive in the same way. It "knows" how to string words together in a persuasive way, but just as some salespeople may not care about how factual their statements are, ChatGPT generates the veneer of realism but will often generate something that in spite of its realism is utter nonsense, or is even mostly "filler words" with little meat. The veneer, filler words and style that humans (and LLMs) use in language are the parts that more similarly mirror the chemical signals used by insects. They are what we use to manipulate and to persuade rather than to convey understanding. This is why LLMs are scary when we consider their use in political misinformation campaigns, but less scary when we consider them writing dissertations about deep scientific topics.
  - epups 4y ago
    > When you say that the output of LLM is "correct", what do you mean? The output that seems impressive is merely "plausibly written by an intelligent entity" but there is nothing about it that could be characterized as being "correct". My point is that we should not make these types of distinctions about how "we" "reason" versus how an LLM does. For example, I have no idea if your reply came from ChatGPT, but it sounds pretty intelligent to me. In that sense, your output is "correct" because it led me to believe I am arguing with an intelligent being. The analogy to a broken clock which is correct twice a day is ridiculous, because ChatGPT, on average, is quite capable of producing meaningful replies to a very complex problem (natural language queries). I don't quite get your analogy to insects. If we figure out an analog pheromone to the ones ants use to communicate, then we found out something very meaningful about ants. Likewise here, I am quite certain ChatGPT is not producing language in the same way a human does, but is it using the same principles? Chomsky and others are prescriptive in saying it is not, yet their arguments are not convincing.
neilv 4y ago
> When I was in grad school, in the early 1990s, a popular sport was “jump on Noam Chomsky”. He gave a series of lectures every year on linguistics and the mind. I went, and so did hundreds of other people. And every week, a bunch of folks would stand up and take cracks at Chomsky, In the late 1990s, some undergrads would see Chomsky campus-wide talks as an opportunity to mock him, but it seemed to be about disagreement with his "politics". "Dept. of Leftist Linguistics", and worse. (The student body seemed overall libertarian, and of overall narrow exposure to the world outside STEM, compared to my previous university.)
robg 4y ago
…these systems are useful technology…” Vs. Therefore they are telling us nothing about language, cognition, acquisition, anything. As anything with Chomsky, the hyperbole misses the point. A useful piece of technology still has uses. An airplane still flies even if it tells us nothing about how birds learn to fly or think about flight or even navigate the globe during an annual migration.
deleted 4y ago
[deleted]
calpusca1 4y ago
I'd like to state hypothesis (sounding like thesis. But please think of it as a start of discussion more than opinion written in stone) that ChatGPT won't produce original work of general physics-science theme because it doesn't have contact with reality. I think ChatGPT is/will be capable of 'understanding' the world/reality in as much as people will write about it in the internet. But by itself it doesn't observe nothing and has no thoughts about the world. If human won't observe something then ChatGPT won't observe it neither. Not my main point but to be more nuanced I think ChatGPT could produce *some* amount of original science in places where people written about some phenomena but weren't able to connect the dots (come up with analogies shedding new light etc.). But ChatGPT won't go out looking for the dots. I think it won't go looking for new dots because of 2 reasons. 1. It can't. It operates on language alone 2. It doesn't ask questions by itself. It only gives answers. It doesn't have the drive to seek knowledge. I'll try to illustrate my intuition: There was a gorilla Koko. She was taught to communicate with humans and could answer questions. But Koko didn't ask questions. She was observing the world but didn't have the concept of unknowns. The sphere of things she didn't know simply wasn't existing for her. I feel like ChatGPT is the same in that regard. 1. I can't find the article. I can be mistaken about Koko, but my illustration still stands. 2. (So many tangents....) Maybe 'curious' model could be built, but ChatGPT would only be a language part of the bigger model. Whole other paradigm would be responsible for curiosity part Back to main point. I have problem with mathematics in the context of ChatGPT coming up with *original* work. I don't know what is the current consensus in philosophy of mathematics about it being discovered or invented. If it's discovered (the fact that abstract mathematics seemingly not describing anything real happen to be used 50 years later as description of some part of reality would suggest that) then my 'not thesis - thesis' stands. It won't produce new original mathematics. If it's invented then it won't need contact with reality and it (theoretically?) could *creatively/originally* advance mathematics (once it already exist). Question. Mathematics originates in axioms. Maybe that's why it fits the world. Because people observing the world chose the axioms that made sense to them. So when it's advacing to it's logical conclusion it still happen to fit the reality. If mathematics wasn't formalized already would ChatGPT ever invent mathematics? Would it come with any kind of axioms?

58 points by bundie 4y ago

Just a bit of context (not related to AI). Chomsky appeared on pro Russian podcasts and expressed some controversial opinions.

unconed 4y ago
Is this a satirical comment about how people try to inject blatant guilt by association under the guise of "context"?
- deleted 4y ago
  [deleted]
- t344344 4y ago
  No, warning readers that Chomsky may be under usual smear campaign.
- hackandthink 4y ago
  A nice reddit thread about some Chomsky critics: https://www.reddit.com/r/chomsky/comments/11gfo8m/why_are_chomsky_haters_so_aggressive/ https://www.reddit.com/r/chomsky/comments/11gfo8m/why_are_ch...

z3c0 4y ago

I too agree with Chomsky. It's concerning to see people piling onto GPT/LLMs as an extractive tool - something it does okay at, but in the end, something it's not made for. So many companies rushing to use it as such is going to result in some colossal problems for some. The potential for completely false assertions is not something one should be in a hurry to productionize. There's already a myriad of extractive NLP solutions available that organizations could be utilizing (like any simpler BERT-based model.) Unfortunately, they don't come with the veneer of misleading ethos that GPT/LLMs pretend to have. Personally, I see some potential for customer-facing uses, but that's because customers like novelties.

bundie 4y ago
Do you think AGI will be achieved if this trend continues?
- z3c0 4y ago
  Not really, no, but that's not to say that LLMs couldn't be a useful shell through which an AGI could interact with the rest of the world. But to herald LLMs as the coming of AGI would be to put the cart before the horse. A rudimentary intelligence capable of basic object-recognition and permanence* would be an actual step towards AGI over LLMs, in my professional (but still fallible) opinion. Something as basic as an insect, or hell, even protozoa, is all we would need to say that we're on the way. Once we have that, tying it with an LLM to give it denotative capabilities would be the boom we're all hoping for/terrified of. [*] this is key, but we don't need to codify this so much as find a way for the object-recognition to build up to it. Even children take a while to develop object permanence (hence peek-a-boo being fun for them - you really are disappearing.)
  - dr_dshiv 4y ago
    I really don’t understand why it isn’t plausible that LLMs are weak AGI. If we’d been wise enough to create benchmarks for them a decade ago, I’m sure they would have exceed our expectations. The generality of the intelligence is unquestionable, at least in a language domain. Discounting it as AGI because a fly is better at being a fly seems besides the point.
    - goatlover 4y ago
      Because they don't do anything but generate text when prompted, while humans are walking around doing things. I would expect at least two things from an AGI, weak or strong: 1. It has agency. 2. It can move about and manipulate the world in a general as opposed to predefined sense.
      - Dzugaru 4y ago

ev7 4y ago

Enjoyed the content of the article, but its credibility is diminished a bit by its numerous typos.

mtlmtlmtlmtl 4y ago
Uhm, how?

hackandthink 4y ago

Non paywall link to Chomsky's essay: https://dnyuz.com/2023/03/08/noam-chomsky-the-false-promise-of-chatgpt/ https://dnyuz.com/2023/03/08/noam-chomsky-the-false-promise-...

greenyoda 4y ago
And the HN discussion of it: https://news.ycombinator.com/item?id=35067619 https://news.ycombinator.com/item?id=35067619

bmitc 4y ago

An observation of mine with regards to ardent Chomsky critics is that it's usually the case that the critic has a lot to gain from their attack or position while Chomsky has little to nothing to gain. I think the whole stable diffusion and ChatGPT thing will come and go, almost as self-driving cars have. If you remember, just eight or so years ago, everybody was absolutely convinced that they were right around the corner. There was an insane amount of hype. Now, that hype has largely come and gone, with several companies abandoning their self-driving approaches or just flat out failing at them. There is still some hype, but I feel it has faded as people have come to grips with the premise, much less the difficulty of the implementations. The focus seems to be shifting back to driver assistance features. Another thing is VR. So these stable diffusion and GPT methods are having their time in the sun. Yet again, just like VR and self-driving, proponents are trying to will these things into existence. Yes, they are impressive in some very specific cases, but they are not general. Just like stable diffusion, these GPT things will just flood the Internet with junk, either filling people with incorrect knowledge or driving people away, leaving the only remaining application to be advertising and trying to sell stuff. Humans are seemingly absolutely obsessed with technology trying to replace humans and reality. That is the common thread between all of VR, self-driving, stable diffusion, and now ChatGPT. But this is both hard and destined for failure. People keep discovering that the role of technology is to augment humans and not to replace them. Engelbart knew this 60 years ago.

mishkovski 4y ago
That is why Apple’s next big product will probably be an application of AR.
goatlover 4y ago
Don't forget the 3D/additive printing revolution that was going to put a printer in everyone's home so they wouldn't need to buy stuff from Walmart anymore. Everyone knows the hype cycle is real, and yet everybody still gets caught up during the early hype.
- code_duck 4y ago
  To me it feels like the late 70s/early 80s and home computing software. These technologies do exist, as far as self driving cars, chatbots that answer all of your questions insightfully and correctly, and 3D printers that can manufacture household goods in demand. There are working examples in use. They’re not quite ready for the mass market, but they will be someday. I’d give it 15-20 years.

331c8c71 4y ago

It's not that I agree with everything Chomsky says or writes but I can't shake the feeling that he's much more intelligent, eloquent and gentlemanly than any of his critics.

the-grump 4y ago
+1 I have my own reservations about his opinion but would never state them in the combative and sneering tone often employed by his critics. Can't help but feel like they personally have a problem with him, and they're using this piece as a cudgel to "take a swing".
tgv 4y ago
Even though I largely disagree with Chomsky, he is extremely intelligent. That Hinton and Sejnowski behave like sore losers is a bit of a disappointment. The argument with the learning disabled student is a disgrace. It's almost as if they cannot imagine that the human brain has innate properties beyond "weight adaptation".
- epups 4y ago
  I did not see a problem with Hinton's argument here - perhaps you would care to expand? If we are evaluating LLM's purely on outcome alone, it seems only fair to compare them to humans that don't have fully functioning cognitive capabilities. We wouldn't discredit a schizophrenic person that hallucinates, we know that this person can also see reality for what it is, most of the time. I just don't think that finding absurd examples, usually after careful prompting, is a slam dunk against LLM's at all.
- naasking 4y ago
  > The argument with the learning disabled student is a disgrace. It's almost as if they cannot imagine that the human brain has innate properties beyond "weight adaptation". They can, it's just that Chomsky's dismissal of "weight adaptations" as insufficient is not actually justified by any evidence. The Transformer model augmented with memory is Turing complete, so it can in principle learn to compute anything using only weight adaptations.

masswerk 4y ago

> To his credit Sejnowski correctly picked on a weak point in the oped: Chomsky’s ChatGPT examples falling apples and gravity were anecdotal and insufficiently nuanced. This, I fail to understand. Clearly, Chomsky started with an exposition of the concept of negativity. How can this be missed by anyone with cursory familiarity with western philosophy? The rest of the article (including the passage regarding the apple) is pretty much a series of excursions on this theme, how LLMs systematically miss negativity.

masswerk 4y ago
It may be observed that Chomskys piece is in this sense much a modern take on Hegel's Phenomenology of Spirit, who also starts in his exposition with the core theme of negativity, as the very base of "sense certainty" (better: sensual certainty, sinnliche Gewissheit), what Hegel called the "teachings of Ceres", the very origin of conceptuality, in order to expand on this in his grand scheme. (I find it quite remarkable that Chomsky would fall back on this classic line of thought on this occasion.)
epups 4y ago
It's pretty bad because Chomsky provided only two examples of things ChatGPT cannot do - even implying its underlying architecture could not ever do that -, and it's obvious that neither he, his coauthors or the editors at the NYT bothered to fact check these assertions. I would bet that Chomsky himself has never played around with ChatGPT itself.
- masswerk 4y ago
  If we subscribe for a moment to the tradition that conceptuality, creativity, even certainty, are born from negativity, that crucial "what if not" assumption, the ponderability of existential alternatives, we may indeed defer that this is not something LLMs (in their current configuration) can do. (In a more statistical tradition, proximity to the semantic center is pretty much opposed to this.) Which poses yet another question, namely, how essential and significant are these things for us? Is the alleged promise aligned to our basic assumptions?

roenxi 4y ago

Chomsky may be right or wrong. The problem the philosophers face is the high likelihood that the "a lumbering statistical engine for pattern matching, gorging on hundreds of terabytes of data and extrapolating" [0] is going to just be more effective than whatever it is human brains do, similar to how rocks turn out to be more effective at adding numbers together than a mathematician despite the early indications from prehistory to whenever the calculator became viable. Statistical models turn out to be enough to get past the Turing test. They're probably going to be better at interpreting vague requirements and making decisions than most humans too. They're better at art than the average, and better at explaining themselves than most humans. [0] https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chatgpt-ai.html https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chat...

apwheele 4y ago
Reminds me a bit of this blog post by Tal Yarkoni, https://www.talyarkoni.org/blog/2018/08/18/if-we-already-understood-the-brain-would-we-even-know-it/ https://www.talyarkoni.org/blog/2018/08/18/if-we-already-und.... That is perhaps more consistent with the "we are just really complicated neural networks argument".
mtlmtlmtlmtl 4y ago
People throw around this claim about the turing test. I've yet to see any empirical corroboration of it. Using an LLM and being amazed by its response is not doing the Turing test. The Turing test is mostly a thought experiment anyway.
- em-bee 4y ago
  if a machine can pass the turing test then that most likely means that the test was not good enough. it is not an indication that the machine is actually intelligent.
  - mtlmtlmtlmtl 4y ago
    I think ChatGPT would fail the test as described by Turing almost immediately, because at some point it would just tell the other test subject it's a language model. In some sense ChatGPT is actually engineered to fail the test.

azubinski 4y ago

Chomsky is saying things that are completely obvious, and that's where the comedy comes in. The tragedy seems to be that during the hype of any technological bubble, someone of the scale of Chomsky is needed to make at least someone hear the obvious. But maybe it's also a comedy. However, it doesn't matter. All flying cars will be on RISK V and Rust and on the blockchain soon. That's what's main.

naasking 4y ago
> Chomsky is saying things that are completely obvious, and that's where the comedy comes in. No, the comedy is that he's saying things that seem completely obvious but are actually completely wrong.

aborsy 4y ago

Aaronson works on quantum complexity theory. He has almost no papers on language or AI (classical one, the one that has produced LLMs in question). He is going to go to OpenAI soon for some sort of position (perhaps PR around quantum ML, that is nevertheless important for the company). Chomsky has around half a million citations to his scientific work, mostly on language. The guy is amazing. At age 94, he is productive, sharp and very well spoken. I tend to take him more seriously on this. One problem is that, experts in one domain opine about domains they have not worked on. Another problem is, some people apparently hold a view that, if you make a product that works and sells, nothing else matters. Everyone will rush to earn a living around it. They feel empowered to take any position. So Science doesn’t matter if it’s not aligned, products do. Aaronson didn’t publish a proper article addressing Chomsky’s arguments.

mudita 4y ago
As far as I know, Scott Aaronson has been on leave from UT Austin to work on theoretical foundations of AI safety at OpenAI since June 2022: https://scottaaronson.blog/?p=6484 https://scottaaronson.blog/?p=6484
- the-grump 4y ago
  That sounds like a conflict of interest. I haven't read his article but would expect this to be noted at the top.
  - mudita 4y ago
    There is a note on Aaronson’s blog post: "[Note for people who might be visiting this blog for the first time: I’m a CS professor at UT Austin, on leave for one year to work at OpenAI on the theoretical foundations of AI safety. I accepted OpenAI’s offer in part because I already held the views here, or something close to them; and given that I could see how large language models were poised to change the world for good and ill, I wanted to be part of the effort to help prevent their misuse. No one at OpenAI asked me to write this or saw it beforehand, and I don’t even know to what extent they agree with it.]"

erwald 4y ago

IMO Chomsky (and Marcus) are mostly wrong on LLMs. Quoting the summary of a post[1] I wrote recently: - LLMs are sometimes said to be “just” shallow pattern matchers, “just” massive look-up tables or “just” autocomplete engines. These comparisons amount to a form of (methodological) reductionism. While there’s some truth to them, I think they smuggle in corollaries that are either false or at least not obviously true. - For example, they seem to imply that what LLMs are doing amounts merely to rote memorisation and/or clever parlour tricks, and that they cannot generalise to out-of-distribution data. In fact, there’s empirical evidence that suggests that LLMs can learn general algorithms and can contain and use representations of the world similar to those we use. - They also seem to suggest that LLMs merely optimise for success on next-token prediction. It’s true that LLMs are (mostly) trained on next-token prediction, and it’s true that this profoundly shapes their output, but we don’t know whether this is how they actually function. We also don’t know what sorts of advanced capabilities can or cannot arise when you train on next-token prediction. - So there’s reason to be cautious when thinking about LLMs. In particular, I think, caution should be exercised (1) when making predictions about what LLMs will or will not in future be capable of and (2) when assuming that such-and-such a thing must or cannot possibly happen inside an LLM. https://www.erichgrunewald.com/posts/against-llm-reductionism/ https://www.erichgrunewald.com/posts/against-llm-reductionis...

resters 4y ago
Think of it this way: They create plausible patterns of language that seem (to humans) similar to human-generated language. That is what they are "good at". Imagine an "AI" that "knows" about rhyme and meter and can generate a seemingly infinite collection of catchy songs that follow the II V I chord progression, only the sequence of words doesn't necessarily make any sense. LLMs are a more powerful version of that kind of thing that does put the words in a sequence that feels like natural human language. Since they are trained on information, the sequences chosen often seem informational, even though they are just patterns. It's really a lot like autocomplete. In other words, a lot of useful information may accidentally be encoded into the model, but so is a lot of stuff that looks/feels similar but is nonsense.
- 4y ago

erwald 4y ago

> Aaronson’s biggest error, now corrected, sort of, in bold, is in assuming that Chomsky has spent his life in some sort of failed effort to build AI, which kind of entirely misses the point of Chomsky’s piece (which says in so many words that we need to study the mind first before we try to make AI) and also utterly misrepresents Chomsky’s career. Frankly, I would be embarrassed to have to publish a correction (the part in bold) like this [...] It’s a wild swing and a miss. Chomsky has spent his career trying to understand how humans acquire language, not “building machines” to try to do the same. This is pretty unfair IMO. Aaronson wrote (emphasis mine): > In this piece Chomsky, THE INTELLECTUAL GODFATHER GOD OF an effort that failed for 60 years to build machines that can converse in ordinary language, condemns the effort that succeeded. He never wrote that Chomsky himself was involved in this efforts, only that he was influential to it (which may or may not be true, but Marcus never argues that point).

epups 4y ago

> If a broken clock were correct twice a day, would we give it credit for patches of understanding of time? If a clock was correct 95% of the time, would you say it has no understanding of time? To me it seems that both Marcus and Chomsky are resorting to a sort of dualistic theory here. They concede that these LLM's are capable of impressive displays of cognition, but wait! It's not real cognition of course, it's fake, unlike the one in humans. Well, we don't know enough about cognition in humans to reject the hypothesis that we are also communicating or thinking based on similar principles. ChatGPT is by far the most successful automaton for understanding and producing language we have ever seen. The language Chomsky uses in the op-ed and in his reply here - "(...)Therefore they are telling us nothing about language, cognition, acquisition, anything" - is pure hubris and completely unjustified.

resters 4y ago
When you say that the output of LLM is "correct", what do you mean? The output that seems impressive is merely "plausibly written by an intelligent entity" but there is nothing about it that could be characterized as being "correct". Similarly, we could sprinkle formic acid on the floor in a pattern that fooled ants into following it in search of food, but there is nothing "correct" about it, merely "realistic". Asking GPT3 to create a persuasive essay is impressive in the same way. It "knows" how to string words together in a persuasive way, but just as some salespeople may not care about how factual their statements are, ChatGPT generates the veneer of realism but will often generate something that in spite of its realism is utter nonsense, or is even mostly "filler words" with little meat. The veneer, filler words and style that humans (and LLMs) use in language are the parts that more similarly mirror the chemical signals used by insects. They are what we use to manipulate and to persuade rather than to convey understanding. This is why LLMs are scary when we consider their use in political misinformation campaigns, but less scary when we consider them writing dissertations about deep scientific topics.
- epups 4y ago
  > When you say that the output of LLM is "correct", what do you mean? The output that seems impressive is merely "plausibly written by an intelligent entity" but there is nothing about it that could be characterized as being "correct". My point is that we should not make these types of distinctions about how "we" "reason" versus how an LLM does. For example, I have no idea if your reply came from ChatGPT, but it sounds pretty intelligent to me. In that sense, your output is "correct" because it led me to believe I am arguing with an intelligent being. The analogy to a broken clock which is correct twice a day is ridiculous, because ChatGPT, on average, is quite capable of producing meaningful replies to a very complex problem (natural language queries). I don't quite get your analogy to insects. If we figure out an analog pheromone to the ones ants use to communicate, then we found out something very meaningful about ants. Likewise here, I am quite certain ChatGPT is not producing language in the same way a human does, but is it using the same principles? Chomsky and others are prescriptive in saying it is not, yet their arguments are not convincing.

neilv 4y ago

> When I was in grad school, in the early 1990s, a popular sport was “jump on Noam Chomsky”. He gave a series of lectures every year on linguistics and the mind. I went, and so did hundreds of other people. And every week, a bunch of folks would stand up and take cracks at Chomsky, In the late 1990s, some undergrads would see Chomsky campus-wide talks as an opportunity to mock him, but it seemed to be about disagreement with his "politics". "Dept. of Leftist Linguistics", and worse. (The student body seemed overall libertarian, and of overall narrow exposure to the world outside STEM, compared to my previous university.)

robg 4y ago

…these systems are useful technology…” Vs. Therefore they are telling us nothing about language, cognition, acquisition, anything. As anything with Chomsky, the hyperbole misses the point. A useful piece of technology still has uses. An airplane still flies even if it tells us nothing about how birds learn to fly or think about flight or even navigate the globe during an annual migration.

deleted 4y ago

[deleted]

calpusca1 4y ago

I'd like to state hypothesis (sounding like thesis. But please think of it as a start of discussion more than opinion written in stone) that ChatGPT won't produce original work of general physics-science theme because it doesn't have contact with reality. I think ChatGPT is/will be capable of 'understanding' the world/reality in as much as people will write about it in the internet. But by itself it doesn't observe nothing and has no thoughts about the world. If human won't observe something then ChatGPT won't observe it neither. Not my main point but to be more nuanced I think ChatGPT could produce *some* amount of original science in places where people written about some phenomena but weren't able to connect the dots (come up with analogies shedding new light etc.). But ChatGPT won't go out looking for the dots. I think it won't go looking for new dots because of 2 reasons. 1. It can't. It operates on language alone 2. It doesn't ask questions by itself. It only gives answers. It doesn't have the drive to seek knowledge. I'll try to illustrate my intuition: There was a gorilla Koko. She was taught to communicate with humans and could answer questions. But Koko didn't ask questions. She was observing the world but didn't have the concept of unknowns. The sphere of things she didn't know simply wasn't existing for her. I feel like ChatGPT is the same in that regard. 1. I can't find the article. I can be mistaken about Koko, but my illustration still stands. 2. (So many tangents....) Maybe 'curious' model could be built, but ChatGPT would only be a language part of the bigger model. Whole other paradigm would be responsible for curiosity part Back to main point. I have problem with mathematics in the context of ChatGPT coming up with *original* work. I don't know what is the current consensus in philosophy of mathematics about it being discovered or invented. If it's discovered (the fact that abstract mathematics seemingly not describing anything real happen to be used 50 years later as description of some part of reality would suggest that) then my 'not thesis - thesis' stands. It won't produce new original mathematics. If it's invented then it won't need contact with reality and it (theoretically?) could *creatively/originally* advance mathematics (once it already exist). Question. Mathematics originates in axioms. Maybe that's why it fits the world. Because people observing the world chose the axioms that made sense to them. So when it's advacing to it's logical conclusion it still happen to fit the reality. If mathematics wasn't formalized already would ChatGPT ever invent mathematics? Would it come with any kind of axioms?