17 ms·

The False Promise of Chomskyism

138 points by nikbackm 4y ago

morelisp 4y ago
> In this piece Chomsky, the intellectual godfather of an effort that failed for 60 years to build machines that can converse in ordinary language At least he leads with the fact he doesn’t understand Chomsky’s research program at all.
- ak_111 4y ago
  Yep, it also makes Chomsky sound bitter because chatgpt "works", but Chomsky's intellectual reputation had very little stake in getting any alternative approach working. A more accurate attack on Chomsky was that his various programs haven't yet achieved to find anything close to a universal grammar, but this is not really related to deep learning working.
ak_111 4y ago
I think comparing Chomsky to Jesuit astronomers declining to look through Galileo’s telescope is an unfair analogy. A better analogy in my opinion would be Einstein's attitude to quantum mechanics, he accepted that it had predictive power but was militantly in the camp that it was incomplete as a theory for various reason. Similar to how Chomsky thinks that deep learning is effective and interesting as an engineering concept to generate language but sheds relatively very little light on how human intelligence and language works on a deeper level.
- eternalban 4y ago
  The scientific community at the time had good reason to resist looking through telescopes. The entire scientific edifice was built on 7 planets going back thousands of years. What would happen to their medical "sciences", for example, if planet #8 showed up? Also note the church had already accepted Coppernicus's findings earlier. Issue was never the caricature of "bible says" that is claimed. In fact it was Babylonian roots of "sciences" -- everything around number 7 and 7 planets -- that caused a foundational problem. They had to rebuild sciences from the ground up. That was the reason for "jesuit" resistance.
  - timeon 4y ago
    This OT debate just shows that using analogy as argument is cul-de-sac.
    - psychlops 4y ago
      That's like saying it's a stalemate.
  - pfortuny 4y ago
    Well, but they did use telescopes. One jesuit even discovered a comet, and it did not show parallax… The jesuits did wrong but were not silly.
    - eternalban 4y ago
      Of course they did. There is no reason to assume people were stupid [in the past] and just got smart overnight. These were smart people struggling with a paradigm shift that would upset almost all their theoretical apple carts. It was an epistomelogical crisis but not of the spiritual flavor as it has been alleged.
- genman 4y ago
  [flagged]
  - kgarten 4y ago
    ha ha ... every human is a fool in that case (as every human holds some unreasonable believes and opinions some point in time). "He has discredited his mind?" This sentence does not make sense to me. Explain what you mean by this statement. I found his NYT piece well argued. How can you fix any language model similar to ChatGPT to prevent the mistakes shown in the article? (I don't think you can).
    - genman 4y ago
      He makes irrelevant and baseless claims. First we don't really know how the mind works and therefore we really have no idea if the language models are path toward AGI or not. He really doesn't have an idea how a child obtains a language. You really need a lot of data to acquire grammar, no minuscule data is not enough. I have observed it personally with my multilingual children, their mother tongue, to what they have the least exposure and what is the most complex, is not still not fluent for them. His claims are simply false, easily refuted by empirical data.
      - kgarten 4y ago
        ok, you didn't answer my first questions. I try a couple of more. your first argument is useless, as anything might be a path to AGI ... first order logic / expert systems are ... regarding empirical data, you are bringing up anecdotes that don't hold up. Humans can often learn a concept from 1-2 instances (take the face of Albert Einstein) (even your kids can do that in grammar :) , maybe check how language models are trained in comparison. do you have any references for your claims? How much data does a human need to learn grammar? do you have any estimate? you say you can refute Chomsky's claims by empirical data. please show me that data/studies, don't talk about anecdotes from your kids.
        genman 4y ago
- calf 4y ago
  Not only the parallels between Einstein's and Chomsky's objections, but the "shut up and calculate" historical camp in quantum mechanics bears some resemblance to perhaps Norvig's arguments for statistical, neural network-based models of computation.
gostsamo 4y ago
> I’ll be busy all day at the Harvard CS department, where I’m giving a quantum talk this afternoon, but for now: Please, make it even more condescending, this is what we need in a good debate. /s
- zitsarethecure 4y ago
  It's a quantum talk. You can only know how long the talk is or what it's about, but not both.
  - auxfil 4y ago
    You should really avoid getting entangled in this kind of humor.
- bee_rider 4y ago
  Quantum talk, so, I guess he’s just promising to provide the smallest possible unit of talk. Quite humble actually. I wonder what he’ll go with. I bet: “I”
timspn 4y ago
Here's Chomsky's essay, for comparison: https://archive.ph/cKVj5 https://archive.ph/cKVj5 Aaronson seems to have built a strawman out of this, for some reason.
raisin_churn 4y ago
This is not a response to the Chomsky piece. The main argument advanced by Chomsky et al is that LLMs are neither AGIs nor are they precursors to what we might consider AGIs, because, among other reasons, LLMs "learn" differently to how humans do, and that difference comes with strict limitations on the upper bounds of what LLMs can achieve. I'm certainly no expert on linguistics or AI/ML, so I don't know about all that, but this blog post avoids engaging with that claim, and opts instead for ad hominem.
- scotty79 4y ago
  > LLMs "learn" differently to how humans do Do they? Personally I can't rule out that of LLM model was trained on all of the language a single human heard/read and produced it wouldn't be able to create next utterance that might be indistinguishable from what that human says.
  - mjburgess 4y ago
    Yes, that's granted. The issue is that indistinguishable isnt good enough. This is the core problem with this schematised (and i think, pseudoscientific) computer science approach to intelligence. Output isnt intelligent. So, for any given output, it could have been created by system A or system B, whose properties could be radically different. It matters why, eg., we get "I hate the rain!" as output. If system-A says it because it: cares, hates, muses, imagines, prefers, intends... then that's radically different than if B does so because, "it's combining a weather API with some internet chat history".
    - scotty79 4y ago
      > Yes, that's granted. The issue is that indistinguishable isnt good enough. It starts to remind me of "Yes! But it doesn't have a soul!"
      - mjburgess 4y ago
        If a digital thermometer reads 100C, connected to a black box, are we thereby required to believe that there's boiling water inside the box? Science doesn't deal with the "indistinguishable". We cannot, on earth, simply distinguish between whether we go around the sun, or the sun goes around the earth. Does the solar system have a soul? The world exists, and it has properties, and those are independent of how dumb apes happen to be and what we are in a position to "distinguish" or otherwise. A system generating text is acting as-if its having its intelligence measured. Each sentence we take to be a symptom of its: having a theory of the enviornment, having something to say about it, having some intention, etc. When I say, "I don't like what you're wearing!" that sentence itself isnt somehow "intelligent". It is only a valid measure of my caring, preferring, speaking, intending, thinking... because that is why i said it. A shredder which happened to assemble those words is likewise not intelligent. This is basic science: measurements arent objects; and measurements have validity criteria which is, at least, the causal properties of the system give rise to those measures. In the case of ChatGPT no relevant properties give rise to its ouptut. Its sentences are not caused by any intelligence, and aren't valid measures of it. There is no boiling water. Your digital thermometer is broken.
dtagames 4y ago
Scott didn't understand Noam's complaints very well. He (and I [0], and many others) are pointing out that LLMs cannot think or perform reasoning or exhibit intelligence. Not now, and not ever, because statistical counting of likely words is not intelligence. Lacking the ability to explain the fundamental reasoning concepts behind one's conclusions is a hallmark of machine "learning," which is why it isn't learning. It's word aggregation. The fact that ChatGPT cannot tell if anything is real or correct or not is not a small thing that one can hand-wave away and say, "Well, maybe tomorrow." The inability to discern truth is built into the LLM method. This is not true of people. Certainly not intelligent ones. ChatGPT cannot even correctly perform math or logic, and those are fundamental to intelligence and intellectual development. Hilariously, we already have many programming languages which can do those things, so that's not much of an advance to rely on software that cannot. Aaronson's premise, that Noam Chomsky has nothing more than "sour grapes" to offer, is really puerile and uninformed. See Chomsky's original article [1] for yourself and then decide who is on the hype bandwagon and who is not. [0] https://medium.com/gitconnected/behind-the-curtain-understanding-the-magic-of-chatgpt-3bbd23f0fbb3 https://medium.com/gitconnected/behind-the-curtain-understan... [1] https://web.archive.org/web/20230308104809/https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chatgpt-ai.html https://web.archive.org/web/20230308104809/https://www.nytim...
- gnramires 4y ago
  > The fact that ChatGPT cannot tell if anything is real or correct or not is not a small thing that one can hand-wave away and say, "Well, maybe tomorrow." The inability to discern truth is built into the LLM method. This is not true of people. Certainly not intelligent ones. To be fair, ChatGPT is not a pure predictive LLM. It also undergoes RLHF, a reinfocement learning step which is much more similar to human learning, and allows it to develop a more human-like character, and seems to give it a pretty good grasp on truth(telling), ethics and context, certainly compared to the behavior of a raw predictive LLM with no prompt engineering. I recommend read this blog post for a more nuanced understanding of LLMs: https://astralcodexten.substack.com/p/janus-simulators https://astralcodexten.substack.com/p/janus-simulators
- esperent 4y ago
  > The fact that ChatGPT cannot tell if anything is real or correct or not is not a small thing that one can hand-wave away I don't disagree with your premise. But this applies equally to human intelligence. There's no way to ever tell if the reality you experience is real or not. This is a fundamental fact of existence that we hand wave away so that we can continue to interact with and learn about the world. My favorite thought experiment on this is Boltzmann Brains. https://en.m.wikipedia.org/wiki/Boltzmann_brain https://en.m.wikipedia.org/wiki/Boltzmann_brain
colesantiago 4y ago
IMO this is a bizarre post by Scott. But not surprising since Scott is now employed at OpenAI. https://scottaaronson.blog/?p=6484 https://scottaaronson.blog/?p=6484
- tromp 4y ago
  Seems worthy of a disclaimer at the bottom of his post...
- andrepd 4y ago
  Ahhhh, okay that explains it. I always liked reading Scott's articles on quantum computing, but this was a highly off-putting piece: riddle with mistakes and misrepresentations, and condescending to boot.
- bee_rider 4y ago
  It reads less like a fully thought out competing position article, and more like an off the cuff thing that someone might throw together on his coffee break because a friend asked him about it. Maybe he forgot that he was posting it to his personal blog and put as much thought into as one might put into, say, a HackerNews comment.
- college_physics 4y ago
  That is ultimately the sad nature of this debate: exaggerating the capabilities of certain algorithms and diminishing the capabilities of the human intellect, for... monetary gain It is deeply immoral and it will rekindle the disturbing image of evil technologists and mad, power hungry scientists at a time where we need as much trust and confidence in the use of technology and science to absolve us of problems we ourselves have created.
- deleted 4y ago
  [deleted]
deleted 4y ago
[deleted]
oldgradstudent 4y ago
> In this piece Chomsky, the intellectual godfather of an effort that failed for 60 years to build machines that can converse in ordinary language, condemns the effort that succeeded. Well, Aaronson may not be the godfather, but he's definitely an influential figure in effort that failed for 45 years to develop magic machines.
- sebzim4500 4y ago
  Sure but if a competing team built a quantum computer that could factorise huge numbers Aaronson would presumably not write a piece in the NYT about how it didn't actually count because <insert vague objection here>.
  - oldgradstudent 4y ago
    Well, for years, he headed a small cottage industry dedicated to claiming D-Wave was not a real quantum computer. He was right it that case, similar to how Chomsky s right in this case.
    - sebzim4500 4y ago
      There is a very important distinction between the two cases. D-Wave have never been able to demonstrate a way in which their technology is actually useful whereas (for better or worse) people are actually using ChatGPT for their jobs/lives and claim it is valuable to them.
      - oldgradstudent 4y ago
        > people are actually using ChatGPT for their jobs/lives and claim it is valuable to them. This is true for spreadsheets, word processors, and hydraulic presses. It does not make them AGIs, or has any bearing to Chomsky's argument (especially since he acknowledged that explicitly).
lkrubner 4y ago
"In this piece Chomsky, the intellectual godfather of an effort that failed for 60 years to build machines that can converse in ordinary language, condemns the effort that succeeded." That's incorrect on a few levels. Aaronson is talking about something else, different from what Chomsky worked on. ChatGPT can put together a statistically likely series of tokens, but ChatGPT doesn't understand the meaning of those tokens, and therefore ChatGPT has no concept of "truth." ChatGPT cannot deliberately lie or tell the truth, it has no understanding of such things. By contrast, Chomsky, for much of his career, was a linguist and he focused on the issues of how is knowledge constructed, and how do we know what truth is, and how does language express this? So Chomsky is pointing out that ChatGPT creates a fascinating illusion of a real conversation, but it isn't exactly what AI researchers were aiming for, for several decades. And it is appropriate that the New York Times would want to publish an essay that speaks clearly about what ChatGPT is and is not, because otherwise there is a risk that the general public will get an over-hyped view of what ChatGPT does.
- gnramires 4y ago
  > therefore ChatGPT has no concept of "truth." ChatGPT cannot deliberately lie or tell the truth, it has no understanding of such things Please see my comment below[1], this is not a fair characterization. ChatGPT is not a pure predictive LLM, it also undergoes RLHF which gives it a much more human-like character, including clearly having a concept of truth within its limits. [1] https://news.ycombinator.com/item?id=35080933 https://news.ycombinator.com/item?id=35080933
  - tpoacher 4y ago
    Indeed. I would say a better criticism wouldn't be that it has no notion of truth, but that it has no ability to reason on the basis of a reinterpretation of what might or should constitute truth. Even if you come up with a fancy prompt that asks it to "assume" certain things, it will still have to base that assumption on things that it already 'knows' to be true, as it already perceives them to be true.
- deleted 4y ago
  [deleted]
mjburgess 4y ago
Scott mischaracterises the reasons, which concern capacities of the system not properties of its output. (1) It lacks the capacity to interpret sentences. It's output possess the property of seeming-to-have-interpreted. (2) It lacks the capacity to learn. Learning is, in part, explaining; it is, in part, causal contact with an environment. It's output has the property of seeming-to-be-generated-by-a-learner. (3) It lacks the capacity to speak either truthfully or otherwise. Lacking the capacity to explain, because at least it lacks the capacity to imagine (counterfactuals), it cannot thereby determine whether a claim accords with the way the world is. It's output has the property: seeming-to-be-true. (4) It lacks the capacity to be moral. For all the above, and more: it lacks the capacity to care. It's output has the property: seeming-to-take-moral-stands. The "Jesuit" here, I fear, is the AI fanatic who have turned their telescopes away from reality (intelligence as it actually exists), and have stolen the lenses to read scripture (intelligence as theorised by partisan academics). One has to agree with Chomsky here at the end, "given the amorality, faux science and linguistic incompetence of these systems, we can only laugh or cry at their popularity." The morality point I think should hit home most extremely: how horrifying to treat generated text output as-if it were constructed by an agent who cares. Apparent properties of the output are not properties of the system; ChatGPT does not think, know, care, intend, speak, commnuicate, etc. One can only "laugh or cry" at how absurd this sales pitch: what a horror to be invited to treat ChatGPT as possessing any of these capacities, simply because correlations across a whole internet of text seems to.
- krona 4y ago
  (1) As it is with humans. It's not possible to attach a debugger to a human brain and determine the nature of understanding; you have to ask the human questions about that understanding and interpret the response yourself. (2) This is a strange one to me. It's a bit like saying an essential quality of a human is to experience the world as a human, and therefore nothing except a human can be human. (3) If only all humans met this arbitrary standard of secular scientific rationalism. (4) I'd say 5% of humanity fail to meet this criteria.
  - mjburgess 4y ago
    (1) You realise that we don't investigate human understanding by looking at input-ouptut text correspondances? (This is quite a funny claim!) The reason we think people have the capacity to interpret sentences is their context-appropriate sensory-motor actions, reasoning, emotion, etc. When I say, "quick! a fire!" people: panic, run, get organised, etc. There are a very very large number of highly complex environmental, interpersonal, sensory-motor, emotional, etc. etc. interactions which follow. ChatGPT not only does not display any, but cannot. This isn't up for debate, the system has no capacity to understand text. If I say, "quick! a fire!" it will not do anything which accords with showing an understanding of that. Worse, as a matter of fact, we know its possible to generate output as-if it did understand. So we (1) have known a mechanism whereby it can fake it; and (2) know that this mechanism is the one its using. (2) All thinking possess intentionality which simply means its about something, and the reason we ask, say, "should i call the police?!" is because we are situated in an environment, which causes us to form mental states about that environment. Any system which emits, "should I call the police?" without having a backing representational state of, eg., the police, care, concern, priority, the environment which requires police, etc. *Does Not Mean* this question. It cannot. (3) The capacity to speak truthfully does not mean you speak truthfully. That capacity is simply that you can form representational states which you can introspect as being accurate or inaccurate. ChatGPT lacks that capacity. It does not form representational (intentional) states when it generates text; each sentence is not caused by such a state. (4) The capacity i named was "caring", pretty much all people care, it's necessary for goal-directed action.
rvz 4y ago
Disclaimer: Scott Aaronson is employed at OpenAI. [0] Which explains why he is now defending the Closed AI snake oil over his quantum computing research and is in fact defending his OpenAI equity like the rest of everyone who recently rushed into the OpenAI hype train scam. His weak response to Chomsky tells us that he has no answer to LLMs like ChatGPTs lack of transparent explainability about the sentences it regurgitates, which is the core reason why LLMs are untrustworthy to which Chomsky repeatedly brings up. But of course he chose not to disclose that in his article so here is the source for his announcement that he joined. [0] [0] https://scottaaronson.blog/?p=6484 https://scottaaronson.blog/?p=6484
z3c0 4y ago
I'm sorry, but if you create an NLP product and Chomsky says you missed the mark, you missed the mark. He practically admits to how far ahead of his time Chomsky was, but then somehow tries to mischaracterize it as a bad thing: In this piece Chomsky, the intellectual godfather of an effort that failed for 60 years to build machines that can converse in ordinary language, condemns the effort that succeeded. Have you had a real conversation with ChatGPT? As in a "trying to extract factual information from another entity" conversation? I'll answer for you: no, because ChatGPT is generative (it's in the name) of apparently valid syntax with consistent meaning, but it's not extractive in nature and does not wield denotation behind-the-scenes. "Converse" means a lot more than "vomit words in a proper order at each other". By that alone, it has not "succeeded" in conversing, just arranging terms convincingly (using Chomsky's theories to do so.) I encourage anybody who believes otherwise to read this paper before continuing to posit that ChatGPT is doing anything more: https://www.philosophy-index.com/russell/on-denoting/Russell_-_On_Denoting.pdf https://www.philosophy-index.com/russell/on-denoting/Russell...
davewritescode 4y ago
ChatGPT is a massive accomplishment but in my personal opinion it feels a lot closer to a search engine with a better search interface and missing data attribution. I've been using ChatGPT in my daily life for months at this point, I use it as a software engineer, particularly when I'm exploring a new programming language and looking for idiomatic ways to express myself. 'What's the Rust way to do X' is something that's all over my ChatGPT history at this point. It's a great way to get help when you're completely lost in a new problem domain. However, my issue is that more than once, ChatGPT has suggested code to me that could likely introduce a security issue in a product had someone been dumb enough to blindly copy and paste. If this were a code snippet on a website, I'd probably drop a comment or just remember to ignore whatever I found on that website in the future. So I get why Chomsky isn't impressed, once accuracy really matters ChatGPT falls on its face. When it comes to generating fluff like marketing copy, jokes and other things that don't require any degree of accuracy, it's amazingly good.
- scotty79 4y ago
  I tried to use ChatGPT to explore parser library for Rust called nom. Nothing worked. Syntax was wrong. Midway I resorted to googling and after a short while I learned why. The library had a major overhaul at one point. They switched from using macros to using functions. ChatGPT didn't notice that and served what was mixture of old and new syntax.
  - davewritescode 4y ago
    Yep, this type of thing happens all the time in ChatGPT. I've more than once had ChatGPT suggest that I use something deprecated and sometimes if you poke it and tell it it'll figure it out but I've seen it make the same mistakes over and over. I was working with JWT tokens in GoLang and the library I was using didn't validate expiration by default. I asked ChatGPT before I dug into the library code and it suggested a solution that that bypassed the built in validation of the JWT library I was using and introduced a fairly big security issue. That was the moment I got a little nervous about folks using ChatGPT.
  - pixl97 4y ago
analog31 4y ago
>>> I submit that, like the Jesuit astronomers declining to look through Galileo’s telescope Completely as an aside, from what I've read, the Catholic astronomers did look through Galileo's scope, and confirmed his observational evidence. They were willing to accept a model of Tycho Brahe, in which the sun goes around the earth and the planets go around the sun. Not needing to look through the scope, to know the truth, was introduced as a hypothetical in arguments. And there were Churchmen who were using the controversy to push the Church towards what we might call a more "conservative" position today. But amongst intellectuals, the debate was over whether the evidence supported the heliocentric theory or not.
littlestymaar 4y ago
I'd like to thank Scott for pointing me to the original Chomsky article which I didn't noticed, because this link is probably the most valuable part of this rant post. (And Chomsky's take on ChatGPT is much better than his recent takes about the Russian invasion of Ukraine…)
- genman 4y ago
  And he still fails making really silly claims like "For instance, a young child acquiring a language is developing — unconsciously, automatically and speedily from minuscule data", yeah, no. I have an multilingual children and the amount of data you actually need to learn a language is definitely not small.
  - inimino 4y ago
    I'm not sure if you're aware of the amount of text used to train these large language models, but in that context, yes it is miniscule.
- sebzim4500 4y ago
  >(And Chomsky's take on ChatGPT is much better than his recent takes about the Russian invasion of Ukraine…) Yes, his article does just about clear the incredibly low bar that you have set for him.
twarge 4y ago
Chomsky's article really seems to show the results of decades of living in an echo chamber. >On the contrary, the human mind is a surprisingly efficient and even elegant system that operates with small amounts of information; True, but then > it seeks not to infer brute correlations among data points but to create explanations. This is such an odd dichotomy, and feels really wrong to me. Explanations are to me just further correlations. > a young child acquiring a language is developing — unconsciously, automatically and speedily from minuscule data — a grammar, a stupendously sophisticated system of logical principles and parameters. Chomsky apparently has children, but clearly may have forgotten what it's like for them to learn language! It's years of trying to communicate, constantly, failing, and learning. Now, go ask ChatGPT to Write an ode to <obscure topic> in iambic pentameter and I bet that it will be as astonishing, creative and fluent as a native speaker.
- Ologn 4y ago
  > Chomsky apparently has children, but clearly may have forgotten what it's like for them to learn language! It's years of trying to communicate, constantly failing, and learning. You are correct that children learning language are constantly failing in some manner. There has been a lot of study of the language acquisition of children. However, the way in which children fail and do not fail is specific. Some things children will fail at a number of times until they learn. On the other hand some rules of grammar, logical principles and parameters children seem to pick up immediately, and in Chomsky's view are even innate. So you can divide theoretically possible failures into two categories - ones theoretically possible that are not made, and ones that are theoretically possible and are made. The first category is what he is talking about. Also - there are different languages with different rules, and people learning a language have to learn the rules of English which are different from Spanish which are different than Japanese. Although in Chomsky's view, these languages are all so similar (compared to say the recursively enumerable language a Turing machine uses), they are virtually identical from a theoretical point of view. Also - children make grammar errors such as first saying "I go to the store" (correct) and then "I goed to the store" (incorrect). These errors were not made by data they acquired! The errors children make are showing they are beginning to understand grammar rules, it's a sign they are going off a series of rules and not just regurgitating data they have acquired.
Ologn 4y ago
At one point in the Times article, Chomsky and his colleagues have ChatGPT make their arguments for them. ChatGPT: "I am not conscious, self-aware, or capable of having personal perspectives. I can provide information and analysis based on the data I have been trained on, but I don't have the ability to form personal opinions or beliefs." ChatGPT itself will tell you the limits of its capability.
- genman 4y ago
  I had multiple conversations with ChatGTP about its nature and its capabilities. As such, it is fairly "self conscious".
PaulHoule 4y ago
The irony of it is that Chomsky’s ideas are the foundation of parsers for programming languages. That is, you can make a link between the semantics of arithmetic and logic and the kind of grammar Chomsky talks about and you have… C, Pascal, Python, etc. And people understand it! The semantics of natural languages is over the experience of an animal and if you think of it as a “language instinct”, that instinct is a peripheral of a animal’s brain which has considerable capability for cognition (particularly in the case of mammals and birds) without language. From that point of view natural language competence is a cherry on top of animal intelligence and you can’t simulate a language-using animal without simulating the rest of the animal. ChatGPT does a surprisingly good job of faking linguistic competence with nothing but language so it certainly looks like a challenge to the Chomskyian point of view but I’d remind you that animals, pre-linguistic children, aphasics are all capable of cognition without language so “language is all you need” is still an incomplete position. (But boy there is that strange fact that transformers work for vision although research may be showing that is accidental?) What does it mean for the mind? A major part of animal behavior involves sequences. Think of tying a knot or a bird migrating, or for that matter serializing and deserializing mental structures into words. In the 1980s I read books about the brain that were lost at sea about how a neural network could model sequences and artificial neural networks have come a long way since then.
- pixl97 4y ago
  >aphasics are all capable of cognition without language so “language is all you need” is still an incomplete position. (But boy there is that strange fact that transformers work for vision although research may be showing that is accidental?) This is one of these places I hope we aren't trapping ourselves in our own thinking like we did with flying. "It's only flying if you flap your wings" could be thought of one of those trapping in 1900, when it turns out that is but one way of flying, and throwing lots of power at rotating blades is another. We may be able to create 'intelligence' from text alone, but I do think that it will need input/output methods of divining that truth, hence simulating the IO functions of the animal, whatever that may look like in the end.
  - PaulHoule 4y ago
pfortuny 4y ago
Really, the jesuits did use telescopes. One wonders at the ignorance that paragraph shows. Come on: Galileo was punished for purely political reasons, with the religious argument as a simple excuse. This is not something obscure.
gbanfalvi 4y ago
I think some of Chomsky's issues with ChatGPT might be misunderstandings and some of it can be iterated on and improved. ChatGPT _is_ a statistical engine returning probabilities -- but there's nothing stopping engineers from changing how it weighs its probabilities to reject obvious falsehoods "the earth is flat" or show a moral bias "doing this will hurt people, I will not engage in it" without convoluted prompts or torturing underpaid Kenyans to label abusive content. One thing I would've like Chomsky to engage more in is comparing how people create new "output" from previous "input" vs. how AIs like ChatGPT do it. He talks about "fake thinking" and "real thinking", but doesn't really go into the "hows" of it (ironically, kinda like how ChatGPT would). I _do_ agree, however, that we're nowhere near AGI and this doesn't bring us closer, but I don't know what would, either. On the topic of Aaronson's blog post, I feel like he didn't understand what Chomsky wrote about or intentionally mischaracterised it, then gave a super childish response. It's very embarrassing.
slibhb 4y ago
I think this response is more or less fair. Chomsky has been extremely derisive towards anyone with whom he disagrees. You can see that in the NYT piece but it's been a decades-long pattern where dissenters are automatically morons. That's not a manner befitting a scientist. I agree that "Chomskyism" has been a "false promise" in some sense. It's not clear to me what Chomsky's linguistics have actually accomplished. Perhaps they gave us a new way to think about language. In that sense, they were a philosophical achievement. But we didn't build anything with them, and that's ultimately the test of science. Meanwhile neural nets got us Google translate and ChatGPT (so far). Now, I don't think philosophy is useless, but Chomsky going after machine learning is a bit like "Jesuit astronomers declining to look through Galileo’s telescope" in that Chomsky is a bit like a philosopher immersed in metaphysics who is dismissive towards empirical scientists who accomplished something much more concrete. Then there's the question of whether ChatGPT is conscious and whether it could lead to AGI. I agree with Chomsky that it's not actually intelligent in some sense. But I'm not sure how much that matters. If you can build a Q&A machine, whether it's intelligent or conscious is an interesting philsophical question but ultimately beside the point. Anyway, that debate has been raging for 50 years (see Searle) and isn't especially interesting at this point. As to whether LLMs can scale into AGIs, I have no idea, that depends on how we define AGI. To me, one of the lessons of ChatGPT is that we don't need consciousness to build useful AIs. Chomsky is a humanist. I believe his critique of behaviorialism was based on humanism: he wanted to place humans in a separate category (as philosophers often do). I think his criticism of ChatGPT is ultimately similar. But the worry is misplaced. ChatGPT is in some sense more human than we are. It isn't some foreign, disembodied intelligence. It's based entirely on text we, as a species, generated and in that sense it's a culmination of human potential.
morsecodist 4y ago
I legitimately scrolled to the top of the post and scrolled back down because I assumed I had missed most of it. The author lists four of Chomsky's points, rebuts only the fourth by saying it was intentional, then insults Chomsky with a vague analogy that could apply to anyone engaged in a scientific debate. Regardless of my stance on the issues this post doesn't seem that good to me.
ordu 4y ago
To my mind Chomsky and Aaronson are talking about different things. Chomski says that LLMs are not AGI and they will never be it. Aaronson says LLMs may be a way to AGI. This ideas do not contradict each other. Chomsky talks that LLMs is not sufficient to create AGI, Aaronson says that LLMs is essential to create AGI. A cause can be essential but not sufficient, nothing wrong with it.
YeGoblynQueenne 4y ago
>> In this piece Chomsky, the intellectual godfather of an effort that failed for 60 years to build machines that can converse in ordinary language, condemns the effort that succeeded. Chomsky never had anything to do with any sort of effort "to build machines that can converse in ordinary language". I have no idea where that came from in Aaronson's article. Chomsky was a great influence to cognitive science, and artificial intelligence, in particular in its early years, but he has nothing to do with chatbots. At. all. That's just plain nuts, to open an article against "Chomskyism" (bad form, that title) with such a complete fabrication. Moreover, Chomsky's influence on AI did very much lead to systems that could "converse in ordinary language" - but with which Chomsky had nothing to do, personally, or as a researcher. The "influence" I mention refers to Chomsky's work on formal grammars, and how they have been used for a long time in NLP to model language. For a couple historically notable examples: ELIZA: Joseph Wizenbaum's chatbot pretending to be a Rogerian psychologist. I think at this point in the conversation, everyone has heard about ELIZA and how it drew people in and made them ascribe human qualities to it, just like ChatGPT and friends. https://en.wikipedia.org/wiki/ELIZA https://en.wikipedia.org/wiki/ELIZA SHRDLU: the doctoral thesis of Terry Winograd, written in 1968-70. SHRDLU was a natural language interface to a virtual robotic arm manipulating coloured bricks in a "brick world". It was based on a natural language grammar created by Winograd, and could support intricate instructions that the system then executed with precision. The wikipedia article on SHRDLU has an example, from which I quote: Person: Is there a large block behind a pyramid? Computer: YES, THREE OF THEM: A LARGE RED ONE, A LARGE GREEN CUBE, AND THE BLUE ONE. Person: Put a small one onto the green cube which supports a pyramid. Computer: OK. Person: Put the littlest pyramid on top of it. Computer: OK. Person: Does the shortest thing the tallest pyramid's support supports support anything green? Computer: YES, THE GREEN PYRAMID. Person: What color is it? Computer: BY "IT", I ASSUME YOU MEAN THE SHORTEST THING THE TALLEST PYRAMID'S SUPPORT SUPPORTS. Computer: RED Person: How many things are on top of green cubes? Computer: I'M NOT SURE WHAT YOU MEAN BY "ON TOP OF "IN THE PHRASE "ON TOP OF GREEN CUBES " . DO YOU MEAN: 1 - DIRECTLY ON THE SURFACE 2 - ANWHERE ON TOP OF ? Person: 2 Computer: THREE OF THEM. https://en.wikipedia.org/wiki/SHRDLU https://en.wikipedia.org/wiki/SHRDLU Now, if you've seen discussions with ChatGPT and earlier large language models you'll know that the above is beyond the capabilities of modern systems; including ones trained specifically to manipulate robotic arms etc. ChatGPT, faced with instructions like the above, will soon start to hallucinate objects that don't exist, then hallucinate moving them, and make a pig's meal of the entire state of the blocks world. SHRDLU, confined as it was in its virtual, toy world, could still follow the instructions of its user with absolute precision. We still have nothing that can repeat this feat. Why? Because "Chomskyism" was abandoned, and everyone turned to statistical NLP, after the AI winter of the '90s crushed AI research funding, that's why.
shrimp_emoji 4y ago
>I’m a CS professor at UT Austin, on leave for one year to work at OpenAI on the theoretical foundations of AI safety. I accepted OpenAI’s offer in part because I already held the views here, or something close to them https://youtu.be/XbzGLdiICk4 https://youtu.be/XbzGLdiICk4
allturtles 4y ago
I didn't think Chomsky's NYT piece was very strongly argued (it seems a mish-mash of different points, some somewhat convincing and others quite unconvincing), but this is at least as bad. It starts off with a complete mischaracterization of Chomsky's research program (to imply that he is a disgruntled sore loser in the race to build machine intelligence) and then continues with a bogus analogy to imply Chomsky is a kind of deluded religious fanatic. Then it concludes with a note complaining that people don't trust the objectivity of the author's opinions now that he's employed by the entity he's defending - well, yes, of course they don't, that's the compromise you made when you decided to take Open AI's money.
jacknews 4y ago
"For instance, a young child acquiring a language is developing — unconsciously, automatically and speedily from minuscule data — a grammar, a stupendously sophisticated system of logical principles and parameters. " LOL, this seems completely wrong to me. I think Chomsky's idea is that humans have some kind of 'grammar engine' that just needs configuring from 'miniscule data' aka many years of listening and practice speaking. No doubt humans have some optimized areas for language, but I doubt it's anything so formal.
dang 4y ago
Related ongoing thread: The False Promise of ChatGPT - https://news.ycombinator.com/item?id=35067619 https://news.ycombinator.com/item?id=35067619 - March 2023 (20 comments)
DiscourseFan 4y ago
Fundementally Chomsky has always argued that language must have some sort of core logic beyond Saussure's division between Signifier and Signified, but so far nobody has been able to prove it. What might be most disturbing to him is that LLMs, which operate purely on mass associations, might be closer to human cognition than a notion of cognitive logic. What if the associations are more primary, and the logic comes later? Just as someone else commented, animals work mostly in cycles, patterns, they don't have logic but the cycles of life are burned into them; there could be some magic key that explains why humans possess logic, but it would not be hidden in biology--more likely, since language is by nature a social phenomenon, it would be an aspect of culture.
lisasays 4y ago
In this piece Chomsky, the intellectual godfather of an effort that failed for 60 years to build machines that can converse in ordinary language. Woah there. When did Chomsky ever take an interest in building machines to do anything? What is he even referring to here? From the get-go, this piece sounds like an ad-hominem slam.
obblekk 4y ago
These are fair criticisms but imo miss Chomsky's much bigger point. From Chomsky's essay, 2 important lines: "Whereas humans are limited in the kinds of explanations we can rationally conjecture, machine learning systems can learn both that the earth is flat and that the earth is round. They trade merely in probabilities that change over time. For this reason, the predictions of machine learning systems will always be superficial and dubious." He's saying we trust humans because they can say things like "I'm pretty sure X because of explanation Y" and under the hood, we process the explanation, form our own probability of X and trust our own computation. But since LLMs cannot provide explanations for their beliefs, humans will never be able to rely on LLMs because the way we actually communicate is through explanations, not probabilities. Chomsky's conclusion: this is a good predictor, but not a human. Where Chomsky is actually wrong is he mixes up how good are you at prediction vs. how well can you convince a human of the prediction vs. how intelligent you are. We humans use a combination of accuracy + convincingness as a marker of intelligence. An AI that was just as accurate but 0 ability to convince a human because 0 ability to produce explanations could still be intelligent. This AI would seem like an alien to us, but an alien that could uncannily beat us at any challenge that requires an understanding of the natural world. In fact, we might never truly understand its internal explanations of the world, but still acknowledge it has them and is good at building new ones internally. Imagine being in a room with a foreign language speaker who beats you at chess. Clearly they have a mental model that works, even if they cannot explain it to you.
wg0 4y ago
Is this complete blog post or got trimmed? Because it ends with a claim about Galileo's telescope thus dismissing the critics via an analogy but doesn't offer much concrete chain of reasoning to reach there.
earfort 4y ago
[flagged]
theknocker 4y ago
[dead]
mikepalmer 4y ago
Regarding Chomsky's characterization of LLMs "gorging on hundreds of terabytes of data" compared to the "miniscule data" required for a child to learn a language: The brain is evolved, so the the "gorging" already happened in animals for 100s of millions of years. The brain has a lot of evolved sequence processing, visual processing, language (the authors are linguists and they admit this though it undercuts their point). Only fine tuning of this pretrained model is needed for a child to grow up speaking, say, English vs. French. This requires only a relatively miniscule amount of data. Moreover, it doesn't matter that LLMs work differently from the human brain. Per Larry Wall, TIMTOWTDI ("There is more than one way to do it").
- morelisp 4y ago
  > 100s of millions of years. Most animals don't acquire language, and the non-humans who are the best candidates for it all appear to have developed it independently; specifically in humans the potential is at most around 10mil years ago since we don't find it in other closely-related primates. Actual language is ~3 orders of magnitude younger, and during most of the time since then humans didn't have such high-density ways to ingest it. As I mentioned elsewhere in the comments, GPT-3 consists of around a million modern-human-years of linguistic intake. It seems to have the strong lead here. > it doesn't matter that LLMs work differently from the human brain. It does if your goal is to learn how human language acquisition works, rather than grind through another trillion in VC cash via your mechanical parrots.
hackandthink 4y ago
This Chomskyism and Chomsky hate is strange: "Yes, they’re wrong, and yes, despite being wrong they’re self-certain, hostile, and smug, and yes I can see this, and yes it angers me" Norvig's essay is less hateful and goes deeper. The Norvig - Chomsky Debate https://news.ycombinator.com/item?id=34857287 https://news.ycombinator.com/item?id=34857287

138 points by nikbackm 4y ago

> In this piece Chomsky, the intellectual godfather of an effort that failed for 60 years to build machines that can converse in ordinary language At least he leads with the fact he doesn’t understand Chomsky’s research program at all.

ak_111 4y ago
Yep, it also makes Chomsky sound bitter because chatgpt "works", but Chomsky's intellectual reputation had very little stake in getting any alternative approach working. A more accurate attack on Chomsky was that his various programs haven't yet achieved to find anything close to a universal grammar, but this is not really related to deep learning working.

ak_111 4y ago

I think comparing Chomsky to Jesuit astronomers declining to look through Galileo’s telescope is an unfair analogy. A better analogy in my opinion would be Einstein's attitude to quantum mechanics, he accepted that it had predictive power but was militantly in the camp that it was incomplete as a theory for various reason. Similar to how Chomsky thinks that deep learning is effective and interesting as an engineering concept to generate language but sheds relatively very little light on how human intelligence and language works on a deeper level.

eternalban 4y ago
The scientific community at the time had good reason to resist looking through telescopes. The entire scientific edifice was built on 7 planets going back thousands of years. What would happen to their medical "sciences", for example, if planet #8 showed up? Also note the church had already accepted Coppernicus's findings earlier. Issue was never the caricature of "bible says" that is claimed. In fact it was Babylonian roots of "sciences" -- everything around number 7 and 7 planets -- that caused a foundational problem. They had to rebuild sciences from the ground up. That was the reason for "jesuit" resistance.
- timeon 4y ago
  This OT debate just shows that using analogy as argument is cul-de-sac.
  - psychlops 4y ago
    That's like saying it's a stalemate.
- pfortuny 4y ago
  Well, but they did use telescopes. One jesuit even discovered a comet, and it did not show parallax… The jesuits did wrong but were not silly.
  - eternalban 4y ago
    Of course they did. There is no reason to assume people were stupid [in the past] and just got smart overnight. These were smart people struggling with a paradigm shift that would upset almost all their theoretical apple carts. It was an epistomelogical crisis but not of the spiritual flavor as it has been alleged.
genman 4y ago
[flagged]
- kgarten 4y ago
  ha ha ... every human is a fool in that case (as every human holds some unreasonable believes and opinions some point in time). "He has discredited his mind?" This sentence does not make sense to me. Explain what you mean by this statement. I found his NYT piece well argued. How can you fix any language model similar to ChatGPT to prevent the mistakes shown in the article? (I don't think you can).
  - genman 4y ago
    He makes irrelevant and baseless claims. First we don't really know how the mind works and therefore we really have no idea if the language models are path toward AGI or not. He really doesn't have an idea how a child obtains a language. You really need a lot of data to acquire grammar, no minuscule data is not enough. I have observed it personally with my multilingual children, their mother tongue, to what they have the least exposure and what is the most complex, is not still not fluent for them. His claims are simply false, easily refuted by empirical data.
    - kgarten 4y ago
      ok, you didn't answer my first questions. I try a couple of more. your first argument is useless, as anything might be a path to AGI ... first order logic / expert systems are ... regarding empirical data, you are bringing up anecdotes that don't hold up. Humans can often learn a concept from 1-2 instances (take the face of Albert Einstein) (even your kids can do that in grammar :) , maybe check how language models are trained in comparison. do you have any references for your claims? How much data does a human need to learn grammar? do you have any estimate? you say you can refute Chomsky's claims by empirical data. please show me that data/studies, don't talk about anecdotes from your kids.
      - genman 4y ago
calf 4y ago
Not only the parallels between Einstein's and Chomsky's objections, but the "shut up and calculate" historical camp in quantum mechanics bears some resemblance to perhaps Norvig's arguments for statistical, neural network-based models of computation.

gostsamo 4y ago

> I’ll be busy all day at the Harvard CS department, where I’m giving a quantum talk this afternoon, but for now: Please, make it even more condescending, this is what we need in a good debate. /s

zitsarethecure 4y ago
It's a quantum talk. You can only know how long the talk is or what it's about, but not both.
- auxfil 4y ago
  You should really avoid getting entangled in this kind of humor.
bee_rider 4y ago
Quantum talk, so, I guess he’s just promising to provide the smallest possible unit of talk. Quite humble actually. I wonder what he’ll go with. I bet: “I”

timspn 4y ago

Here's Chomsky's essay, for comparison: https://archive.ph/cKVj5 https://archive.ph/cKVj5 Aaronson seems to have built a strawman out of this, for some reason.

raisin_churn 4y ago

This is not a response to the Chomsky piece. The main argument advanced by Chomsky et al is that LLMs are neither AGIs nor are they precursors to what we might consider AGIs, because, among other reasons, LLMs "learn" differently to how humans do, and that difference comes with strict limitations on the upper bounds of what LLMs can achieve. I'm certainly no expert on linguistics or AI/ML, so I don't know about all that, but this blog post avoids engaging with that claim, and opts instead for ad hominem.

scotty79 4y ago
> LLMs "learn" differently to how humans do Do they? Personally I can't rule out that of LLM model was trained on all of the language a single human heard/read and produced it wouldn't be able to create next utterance that might be indistinguishable from what that human says.
- mjburgess 4y ago
  Yes, that's granted. The issue is that indistinguishable isnt good enough. This is the core problem with this schematised (and i think, pseudoscientific) computer science approach to intelligence. Output isnt intelligent. So, for any given output, it could have been created by system A or system B, whose properties could be radically different. It matters why, eg., we get "I hate the rain!" as output. If system-A says it because it: cares, hates, muses, imagines, prefers, intends... then that's radically different than if B does so because, "it's combining a weather API with some internet chat history".
  - scotty79 4y ago
    > Yes, that's granted. The issue is that indistinguishable isnt good enough. It starts to remind me of "Yes! But it doesn't have a soul!"
    - mjburgess 4y ago
      If a digital thermometer reads 100C, connected to a black box, are we thereby required to believe that there's boiling water inside the box? Science doesn't deal with the "indistinguishable". We cannot, on earth, simply distinguish between whether we go around the sun, or the sun goes around the earth. Does the solar system have a soul? The world exists, and it has properties, and those are independent of how dumb apes happen to be and what we are in a position to "distinguish" or otherwise. A system generating text is acting as-if its having its intelligence measured. Each sentence we take to be a symptom of its: having a theory of the enviornment, having something to say about it, having some intention, etc. When I say, "I don't like what you're wearing!" that sentence itself isnt somehow "intelligent". It is only a valid measure of my caring, preferring, speaking, intending, thinking... because that is why i said it. A shredder which happened to assemble those words is likewise not intelligent. This is basic science: measurements arent objects; and measurements have validity criteria which is, at least, the causal properties of the system give rise to those measures. In the case of ChatGPT no relevant properties give rise to its ouptut. Its sentences are not caused by any intelligence, and aren't valid measures of it. There is no boiling water. Your digital thermometer is broken.

dtagames 4y ago

Scott didn't understand Noam's complaints very well. He (and I [0], and many others) are pointing out that LLMs cannot think or perform reasoning or exhibit intelligence. Not now, and not ever, because statistical counting of likely words is not intelligence. Lacking the ability to explain the fundamental reasoning concepts behind one's conclusions is a hallmark of machine "learning," which is why it isn't learning. It's word aggregation. The fact that ChatGPT cannot tell if anything is real or correct or not is not a small thing that one can hand-wave away and say, "Well, maybe tomorrow." The inability to discern truth is built into the LLM method. This is not true of people. Certainly not intelligent ones. ChatGPT cannot even correctly perform math or logic, and those are fundamental to intelligence and intellectual development. Hilariously, we already have many programming languages which can do those things, so that's not much of an advance to rely on software that cannot. Aaronson's premise, that Noam Chomsky has nothing more than "sour grapes" to offer, is really puerile and uninformed. See Chomsky's original article [1] for yourself and then decide who is on the hype bandwagon and who is not. [0] https://medium.com/gitconnected/behind-the-curtain-understanding-the-magic-of-chatgpt-3bbd23f0fbb3 https://medium.com/gitconnected/behind-the-curtain-understan... [1] https://web.archive.org/web/20230308104809/https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chatgpt-ai.html https://web.archive.org/web/20230308104809/https://www.nytim...

gnramires 4y ago
> The fact that ChatGPT cannot tell if anything is real or correct or not is not a small thing that one can hand-wave away and say, "Well, maybe tomorrow." The inability to discern truth is built into the LLM method. This is not true of people. Certainly not intelligent ones. To be fair, ChatGPT is not a pure predictive LLM. It also undergoes RLHF, a reinfocement learning step which is much more similar to human learning, and allows it to develop a more human-like character, and seems to give it a pretty good grasp on truth(telling), ethics and context, certainly compared to the behavior of a raw predictive LLM with no prompt engineering. I recommend read this blog post for a more nuanced understanding of LLMs: https://astralcodexten.substack.com/p/janus-simulators https://astralcodexten.substack.com/p/janus-simulators
esperent 4y ago
> The fact that ChatGPT cannot tell if anything is real or correct or not is not a small thing that one can hand-wave away I don't disagree with your premise. But this applies equally to human intelligence. There's no way to ever tell if the reality you experience is real or not. This is a fundamental fact of existence that we hand wave away so that we can continue to interact with and learn about the world. My favorite thought experiment on this is Boltzmann Brains. https://en.m.wikipedia.org/wiki/Boltzmann_brain https://en.m.wikipedia.org/wiki/Boltzmann_brain

colesantiago 4y ago

IMO this is a bizarre post by Scott. But not surprising since Scott is now employed at OpenAI. https://scottaaronson.blog/?p=6484 https://scottaaronson.blog/?p=6484

tromp 4y ago
Seems worthy of a disclaimer at the bottom of his post...
andrepd 4y ago
Ahhhh, okay that explains it. I always liked reading Scott's articles on quantum computing, but this was a highly off-putting piece: riddle with mistakes and misrepresentations, and condescending to boot.
bee_rider 4y ago
It reads less like a fully thought out competing position article, and more like an off the cuff thing that someone might throw together on his coffee break because a friend asked him about it. Maybe he forgot that he was posting it to his personal blog and put as much thought into as one might put into, say, a HackerNews comment.
college_physics 4y ago
That is ultimately the sad nature of this debate: exaggerating the capabilities of certain algorithms and diminishing the capabilities of the human intellect, for... monetary gain It is deeply immoral and it will rekindle the disturbing image of evil technologists and mad, power hungry scientists at a time where we need as much trust and confidence in the use of technology and science to absolve us of problems we ourselves have created.
deleted 4y ago
[deleted]

deleted 4y ago

[deleted]

oldgradstudent 4y ago

> In this piece Chomsky, the intellectual godfather of an effort that failed for 60 years to build machines that can converse in ordinary language, condemns the effort that succeeded. Well, Aaronson may not be the godfather, but he's definitely an influential figure in effort that failed for 45 years to develop magic machines.

sebzim4500 4y ago
Sure but if a competing team built a quantum computer that could factorise huge numbers Aaronson would presumably not write a piece in the NYT about how it didn't actually count because <insert vague objection here>.
- oldgradstudent 4y ago
  Well, for years, he headed a small cottage industry dedicated to claiming D-Wave was not a real quantum computer. He was right it that case, similar to how Chomsky s right in this case.
  - sebzim4500 4y ago
    There is a very important distinction between the two cases. D-Wave have never been able to demonstrate a way in which their technology is actually useful whereas (for better or worse) people are actually using ChatGPT for their jobs/lives and claim it is valuable to them.
    - oldgradstudent 4y ago
      > people are actually using ChatGPT for their jobs/lives and claim it is valuable to them. This is true for spreadsheets, word processors, and hydraulic presses. It does not make them AGIs, or has any bearing to Chomsky's argument (especially since he acknowledged that explicitly).

lkrubner 4y ago

"In this piece Chomsky, the intellectual godfather of an effort that failed for 60 years to build machines that can converse in ordinary language, condemns the effort that succeeded." That's incorrect on a few levels. Aaronson is talking about something else, different from what Chomsky worked on. ChatGPT can put together a statistically likely series of tokens, but ChatGPT doesn't understand the meaning of those tokens, and therefore ChatGPT has no concept of "truth." ChatGPT cannot deliberately lie or tell the truth, it has no understanding of such things. By contrast, Chomsky, for much of his career, was a linguist and he focused on the issues of how is knowledge constructed, and how do we know what truth is, and how does language express this? So Chomsky is pointing out that ChatGPT creates a fascinating illusion of a real conversation, but it isn't exactly what AI researchers were aiming for, for several decades. And it is appropriate that the New York Times would want to publish an essay that speaks clearly about what ChatGPT is and is not, because otherwise there is a risk that the general public will get an over-hyped view of what ChatGPT does.

gnramires 4y ago
> therefore ChatGPT has no concept of "truth." ChatGPT cannot deliberately lie or tell the truth, it has no understanding of such things Please see my comment below[1], this is not a fair characterization. ChatGPT is not a pure predictive LLM, it also undergoes RLHF which gives it a much more human-like character, including clearly having a concept of truth within its limits. [1] https://news.ycombinator.com/item?id=35080933 https://news.ycombinator.com/item?id=35080933
- tpoacher 4y ago
  Indeed. I would say a better criticism wouldn't be that it has no notion of truth, but that it has no ability to reason on the basis of a reinterpretation of what might or should constitute truth. Even if you come up with a fancy prompt that asks it to "assume" certain things, it will still have to base that assumption on things that it already 'knows' to be true, as it already perceives them to be true.
deleted 4y ago
[deleted]

mjburgess 4y ago

Scott mischaracterises the reasons, which concern capacities of the system not properties of its output. (1) It lacks the capacity to interpret sentences. It's output possess the property of seeming-to-have-interpreted. (2) It lacks the capacity to learn. Learning is, in part, explaining; it is, in part, causal contact with an environment. It's output has the property of seeming-to-be-generated-by-a-learner. (3) It lacks the capacity to speak either truthfully or otherwise. Lacking the capacity to explain, because at least it lacks the capacity to imagine (counterfactuals), it cannot thereby determine whether a claim accords with the way the world is. It's output has the property: seeming-to-be-true. (4) It lacks the capacity to be moral. For all the above, and more: it lacks the capacity to care. It's output has the property: seeming-to-take-moral-stands. The "Jesuit" here, I fear, is the AI fanatic who have turned their telescopes away from reality (intelligence as it actually exists), and have stolen the lenses to read scripture (intelligence as theorised by partisan academics). One has to agree with Chomsky here at the end, "given the amorality, faux science and linguistic incompetence of these systems, we can only laugh or cry at their popularity." The morality point I think should hit home most extremely: how horrifying to treat generated text output as-if it were constructed by an agent who cares. Apparent properties of the output are not properties of the system; ChatGPT does not think, know, care, intend, speak, commnuicate, etc. One can only "laugh or cry" at how absurd this sales pitch: what a horror to be invited to treat ChatGPT as possessing any of these capacities, simply because correlations across a whole internet of text seems to.

krona 4y ago
(1) As it is with humans. It's not possible to attach a debugger to a human brain and determine the nature of understanding; you have to ask the human questions about that understanding and interpret the response yourself. (2) This is a strange one to me. It's a bit like saying an essential quality of a human is to experience the world as a human, and therefore nothing except a human can be human. (3) If only all humans met this arbitrary standard of secular scientific rationalism. (4) I'd say 5% of humanity fail to meet this criteria.
- mjburgess 4y ago
  (1) You realise that we don't investigate human understanding by looking at input-ouptut text correspondances? (This is quite a funny claim!) The reason we think people have the capacity to interpret sentences is their context-appropriate sensory-motor actions, reasoning, emotion, etc. When I say, "quick! a fire!" people: panic, run, get organised, etc. There are a very very large number of highly complex environmental, interpersonal, sensory-motor, emotional, etc. etc. interactions which follow. ChatGPT not only does not display any, but cannot. This isn't up for debate, the system has no capacity to understand text. If I say, "quick! a fire!" it will not do anything which accords with showing an understanding of that. Worse, as a matter of fact, we know its possible to generate output as-if it did understand. So we (1) have known a mechanism whereby it can fake it; and (2) know that this mechanism is the one its using. (2) All thinking possess intentionality which simply means its about something, and the reason we ask, say, "should i call the police?!" is because we are situated in an environment, which causes us to form mental states about that environment. Any system which emits, "should I call the police?" without having a backing representational state of, eg., the police, care, concern, priority, the environment which requires police, etc. *Does Not Mean* this question. It cannot. (3) The capacity to speak truthfully does not mean you speak truthfully. That capacity is simply that you can form representational states which you can introspect as being accurate or inaccurate. ChatGPT lacks that capacity. It does not form representational (intentional) states when it generates text; each sentence is not caused by such a state. (4) The capacity i named was "caring", pretty much all people care, it's necessary for goal-directed action.

rvz 4y ago

Disclaimer: Scott Aaronson is employed at OpenAI. [0] Which explains why he is now defending the Closed AI snake oil over his quantum computing research and is in fact defending his OpenAI equity like the rest of everyone who recently rushed into the OpenAI hype train scam. His weak response to Chomsky tells us that he has no answer to LLMs like ChatGPTs lack of transparent explainability about the sentences it regurgitates, which is the core reason why LLMs are untrustworthy to which Chomsky repeatedly brings up. But of course he chose not to disclose that in his article so here is the source for his announcement that he joined. [0] [0] https://scottaaronson.blog/?p=6484 https://scottaaronson.blog/?p=6484

z3c0 4y ago

I'm sorry, but if you create an NLP product and Chomsky says you missed the mark, you missed the mark. He practically admits to how far ahead of his time Chomsky was, but then somehow tries to mischaracterize it as a bad thing: In this piece Chomsky, the intellectual godfather of an effort that failed for 60 years to build machines that can converse in ordinary language, condemns the effort that succeeded. Have you had a real conversation with ChatGPT? As in a "trying to extract factual information from another entity" conversation? I'll answer for you: no, because ChatGPT is generative (it's in the name) of apparently valid syntax with consistent meaning, but it's not extractive in nature and does not wield denotation behind-the-scenes. "Converse" means a lot more than "vomit words in a proper order at each other". By that alone, it has not "succeeded" in conversing, just arranging terms convincingly (using Chomsky's theories to do so.) I encourage anybody who believes otherwise to read this paper before continuing to posit that ChatGPT is doing anything more: https://www.philosophy-index.com/russell/on-denoting/Russell_-_On_Denoting.pdf https://www.philosophy-index.com/russell/on-denoting/Russell...

davewritescode 4y ago

ChatGPT is a massive accomplishment but in my personal opinion it feels a lot closer to a search engine with a better search interface and missing data attribution. I've been using ChatGPT in my daily life for months at this point, I use it as a software engineer, particularly when I'm exploring a new programming language and looking for idiomatic ways to express myself. 'What's the Rust way to do X' is something that's all over my ChatGPT history at this point. It's a great way to get help when you're completely lost in a new problem domain. However, my issue is that more than once, ChatGPT has suggested code to me that could likely introduce a security issue in a product had someone been dumb enough to blindly copy and paste. If this were a code snippet on a website, I'd probably drop a comment or just remember to ignore whatever I found on that website in the future. So I get why Chomsky isn't impressed, once accuracy really matters ChatGPT falls on its face. When it comes to generating fluff like marketing copy, jokes and other things that don't require any degree of accuracy, it's amazingly good.

scotty79 4y ago
I tried to use ChatGPT to explore parser library for Rust called nom. Nothing worked. Syntax was wrong. Midway I resorted to googling and after a short while I learned why. The library had a major overhaul at one point. They switched from using macros to using functions. ChatGPT didn't notice that and served what was mixture of old and new syntax.
- davewritescode 4y ago
  Yep, this type of thing happens all the time in ChatGPT. I've more than once had ChatGPT suggest that I use something deprecated and sometimes if you poke it and tell it it'll figure it out but I've seen it make the same mistakes over and over. I was working with JWT tokens in GoLang and the library I was using didn't validate expiration by default. I asked ChatGPT before I dug into the library code and it suggested a solution that that bypassed the built in validation of the JWT library I was using and introduced a fairly big security issue. That was the moment I got a little nervous about folks using ChatGPT.
- pixl97 4y ago

analog31 4y ago

>>> I submit that, like the Jesuit astronomers declining to look through Galileo’s telescope Completely as an aside, from what I've read, the Catholic astronomers did look through Galileo's scope, and confirmed his observational evidence. They were willing to accept a model of Tycho Brahe, in which the sun goes around the earth and the planets go around the sun. Not needing to look through the scope, to know the truth, was introduced as a hypothetical in arguments. And there were Churchmen who were using the controversy to push the Church towards what we might call a more "conservative" position today. But amongst intellectuals, the debate was over whether the evidence supported the heliocentric theory or not.

littlestymaar 4y ago

I'd like to thank Scott for pointing me to the original Chomsky article which I didn't noticed, because this link is probably the most valuable part of this rant post. (And Chomsky's take on ChatGPT is much better than his recent takes about the Russian invasion of Ukraine…)

genman 4y ago
And he still fails making really silly claims like "For instance, a young child acquiring a language is developing — unconsciously, automatically and speedily from minuscule data", yeah, no. I have an multilingual children and the amount of data you actually need to learn a language is definitely not small.
- inimino 4y ago
  I'm not sure if you're aware of the amount of text used to train these large language models, but in that context, yes it is miniscule.
sebzim4500 4y ago
>(And Chomsky's take on ChatGPT is much better than his recent takes about the Russian invasion of Ukraine…) Yes, his article does just about clear the incredibly low bar that you have set for him.

twarge 4y ago

Chomsky's article really seems to show the results of decades of living in an echo chamber. >On the contrary, the human mind is a surprisingly efficient and even elegant system that operates with small amounts of information; True, but then > it seeks not to infer brute correlations among data points but to create explanations. This is such an odd dichotomy, and feels really wrong to me. Explanations are to me just further correlations. > a young child acquiring a language is developing — unconsciously, automatically and speedily from minuscule data — a grammar, a stupendously sophisticated system of logical principles and parameters. Chomsky apparently has children, but clearly may have forgotten what it's like for them to learn language! It's years of trying to communicate, constantly, failing, and learning. Now, go ask ChatGPT to Write an ode to <obscure topic> in iambic pentameter and I bet that it will be as astonishing, creative and fluent as a native speaker.

Ologn 4y ago
> Chomsky apparently has children, but clearly may have forgotten what it's like for them to learn language! It's years of trying to communicate, constantly failing, and learning. You are correct that children learning language are constantly failing in some manner. There has been a lot of study of the language acquisition of children. However, the way in which children fail and do not fail is specific. Some things children will fail at a number of times until they learn. On the other hand some rules of grammar, logical principles and parameters children seem to pick up immediately, and in Chomsky's view are even innate. So you can divide theoretically possible failures into two categories - ones theoretically possible that are not made, and ones that are theoretically possible and are made. The first category is what he is talking about. Also - there are different languages with different rules, and people learning a language have to learn the rules of English which are different from Spanish which are different than Japanese. Although in Chomsky's view, these languages are all so similar (compared to say the recursively enumerable language a Turing machine uses), they are virtually identical from a theoretical point of view. Also - children make grammar errors such as first saying "I go to the store" (correct) and then "I goed to the store" (incorrect). These errors were not made by data they acquired! The errors children make are showing they are beginning to understand grammar rules, it's a sign they are going off a series of rules and not just regurgitating data they have acquired.

Ologn 4y ago

At one point in the Times article, Chomsky and his colleagues have ChatGPT make their arguments for them. ChatGPT: "I am not conscious, self-aware, or capable of having personal perspectives. I can provide information and analysis based on the data I have been trained on, but I don't have the ability to form personal opinions or beliefs." ChatGPT itself will tell you the limits of its capability.

genman 4y ago
I had multiple conversations with ChatGTP about its nature and its capabilities. As such, it is fairly "self conscious".

PaulHoule 4y ago

The irony of it is that Chomsky’s ideas are the foundation of parsers for programming languages. That is, you can make a link between the semantics of arithmetic and logic and the kind of grammar Chomsky talks about and you have… C, Pascal, Python, etc. And people understand it! The semantics of natural languages is over the experience of an animal and if you think of it as a “language instinct”, that instinct is a peripheral of a animal’s brain which has considerable capability for cognition (particularly in the case of mammals and birds) without language. From that point of view natural language competence is a cherry on top of animal intelligence and you can’t simulate a language-using animal without simulating the rest of the animal. ChatGPT does a surprisingly good job of faking linguistic competence with nothing but language so it certainly looks like a challenge to the Chomskyian point of view but I’d remind you that animals, pre-linguistic children, aphasics are all capable of cognition without language so “language is all you need” is still an incomplete position. (But boy there is that strange fact that transformers work for vision although research may be showing that is accidental?) What does it mean for the mind? A major part of animal behavior involves sequences. Think of tying a knot or a bird migrating, or for that matter serializing and deserializing mental structures into words. In the 1980s I read books about the brain that were lost at sea about how a neural network could model sequences and artificial neural networks have come a long way since then.

pixl97 4y ago
>aphasics are all capable of cognition without language so “language is all you need” is still an incomplete position. (But boy there is that strange fact that transformers work for vision although research may be showing that is accidental?) This is one of these places I hope we aren't trapping ourselves in our own thinking like we did with flying. "It's only flying if you flap your wings" could be thought of one of those trapping in 1900, when it turns out that is but one way of flying, and throwing lots of power at rotating blades is another. We may be able to create 'intelligence' from text alone, but I do think that it will need input/output methods of divining that truth, hence simulating the IO functions of the animal, whatever that may look like in the end.
- PaulHoule 4y ago

pfortuny 4y ago

Really, the jesuits did use telescopes. One wonders at the ignorance that paragraph shows. Come on: Galileo was punished for purely political reasons, with the religious argument as a simple excuse. This is not something obscure.

gbanfalvi 4y ago

I think some of Chomsky's issues with ChatGPT might be misunderstandings and some of it can be iterated on and improved. ChatGPT _is_ a statistical engine returning probabilities -- but there's nothing stopping engineers from changing how it weighs its probabilities to reject obvious falsehoods "the earth is flat" or show a moral bias "doing this will hurt people, I will not engage in it" without convoluted prompts or torturing underpaid Kenyans to label abusive content. One thing I would've like Chomsky to engage more in is comparing how people create new "output" from previous "input" vs. how AIs like ChatGPT do it. He talks about "fake thinking" and "real thinking", but doesn't really go into the "hows" of it (ironically, kinda like how ChatGPT would). I _do_ agree, however, that we're nowhere near AGI and this doesn't bring us closer, but I don't know what would, either. On the topic of Aaronson's blog post, I feel like he didn't understand what Chomsky wrote about or intentionally mischaracterised it, then gave a super childish response. It's very embarrassing.

slibhb 4y ago

I think this response is more or less fair. Chomsky has been extremely derisive towards anyone with whom he disagrees. You can see that in the NYT piece but it's been a decades-long pattern where dissenters are automatically morons. That's not a manner befitting a scientist. I agree that "Chomskyism" has been a "false promise" in some sense. It's not clear to me what Chomsky's linguistics have actually accomplished. Perhaps they gave us a new way to think about language. In that sense, they were a philosophical achievement. But we didn't build anything with them, and that's ultimately the test of science. Meanwhile neural nets got us Google translate and ChatGPT (so far). Now, I don't think philosophy is useless, but Chomsky going after machine learning is a bit like "Jesuit astronomers declining to look through Galileo’s telescope" in that Chomsky is a bit like a philosopher immersed in metaphysics who is dismissive towards empirical scientists who accomplished something much more concrete. Then there's the question of whether ChatGPT is conscious and whether it could lead to AGI. I agree with Chomsky that it's not actually intelligent in some sense. But I'm not sure how much that matters. If you can build a Q&A machine, whether it's intelligent or conscious is an interesting philsophical question but ultimately beside the point. Anyway, that debate has been raging for 50 years (see Searle) and isn't especially interesting at this point. As to whether LLMs can scale into AGIs, I have no idea, that depends on how we define AGI. To me, one of the lessons of ChatGPT is that we don't need consciousness to build useful AIs. Chomsky is a humanist. I believe his critique of behaviorialism was based on humanism: he wanted to place humans in a separate category (as philosophers often do). I think his criticism of ChatGPT is ultimately similar. But the worry is misplaced. ChatGPT is in some sense more human than we are. It isn't some foreign, disembodied intelligence. It's based entirely on text we, as a species, generated and in that sense it's a culmination of human potential.

morsecodist 4y ago

I legitimately scrolled to the top of the post and scrolled back down because I assumed I had missed most of it. The author lists four of Chomsky's points, rebuts only the fourth by saying it was intentional, then insults Chomsky with a vague analogy that could apply to anyone engaged in a scientific debate. Regardless of my stance on the issues this post doesn't seem that good to me.

ordu 4y ago

To my mind Chomsky and Aaronson are talking about different things. Chomski says that LLMs are not AGI and they will never be it. Aaronson says LLMs may be a way to AGI. This ideas do not contradict each other. Chomsky talks that LLMs is not sufficient to create AGI, Aaronson says that LLMs is essential to create AGI. A cause can be essential but not sufficient, nothing wrong with it.

YeGoblynQueenne 4y ago

>> In this piece Chomsky, the intellectual godfather of an effort that failed for 60 years to build machines that can converse in ordinary language, condemns the effort that succeeded. Chomsky never had anything to do with any sort of effort "to build machines that can converse in ordinary language". I have no idea where that came from in Aaronson's article. Chomsky was a great influence to cognitive science, and artificial intelligence, in particular in its early years, but he has nothing to do with chatbots. At. all. That's just plain nuts, to open an article against "Chomskyism" (bad form, that title) with such a complete fabrication. Moreover, Chomsky's influence on AI did very much lead to systems that could "converse in ordinary language" - but with which Chomsky had nothing to do, personally, or as a researcher. The "influence" I mention refers to Chomsky's work on formal grammars, and how they have been used for a long time in NLP to model language. For a couple historically notable examples: ELIZA: Joseph Wizenbaum's chatbot pretending to be a Rogerian psychologist. I think at this point in the conversation, everyone has heard about ELIZA and how it drew people in and made them ascribe human qualities to it, just like ChatGPT and friends. https://en.wikipedia.org/wiki/ELIZA https://en.wikipedia.org/wiki/ELIZA SHRDLU: the doctoral thesis of Terry Winograd, written in 1968-70. SHRDLU was a natural language interface to a virtual robotic arm manipulating coloured bricks in a "brick world". It was based on a natural language grammar created by Winograd, and could support intricate instructions that the system then executed with precision. The wikipedia article on SHRDLU has an example, from which I quote: Person: Is there a large block behind a pyramid? Computer: YES, THREE OF THEM: A LARGE RED ONE, A LARGE GREEN CUBE, AND THE BLUE ONE. Person: Put a small one onto the green cube which supports a pyramid. Computer: OK. Person: Put the littlest pyramid on top of it. Computer: OK. Person: Does the shortest thing the tallest pyramid's support supports support anything green? Computer: YES, THE GREEN PYRAMID. Person: What color is it? Computer: BY "IT", I ASSUME YOU MEAN THE SHORTEST THING THE TALLEST PYRAMID'S SUPPORT SUPPORTS. Computer: RED Person: How many things are on top of green cubes? Computer: I'M NOT SURE WHAT YOU MEAN BY "ON TOP OF "IN THE PHRASE "ON TOP OF GREEN CUBES " . DO YOU MEAN: 1 - DIRECTLY ON THE SURFACE 2 - ANWHERE ON TOP OF ? Person: 2 Computer: THREE OF THEM. https://en.wikipedia.org/wiki/SHRDLU https://en.wikipedia.org/wiki/SHRDLU Now, if you've seen discussions with ChatGPT and earlier large language models you'll know that the above is beyond the capabilities of modern systems; including ones trained specifically to manipulate robotic arms etc. ChatGPT, faced with instructions like the above, will soon start to hallucinate objects that don't exist, then hallucinate moving them, and make a pig's meal of the entire state of the blocks world. SHRDLU, confined as it was in its virtual, toy world, could still follow the instructions of its user with absolute precision. We still have nothing that can repeat this feat. Why? Because "Chomskyism" was abandoned, and everyone turned to statistical NLP, after the AI winter of the '90s crushed AI research funding, that's why.

shrimp_emoji 4y ago

>I’m a CS professor at UT Austin, on leave for one year to work at OpenAI on the theoretical foundations of AI safety. I accepted OpenAI’s offer in part because I already held the views here, or something close to them https://youtu.be/XbzGLdiICk4 https://youtu.be/XbzGLdiICk4

allturtles 4y ago

I didn't think Chomsky's NYT piece was very strongly argued (it seems a mish-mash of different points, some somewhat convincing and others quite unconvincing), but this is at least as bad. It starts off with a complete mischaracterization of Chomsky's research program (to imply that he is a disgruntled sore loser in the race to build machine intelligence) and then continues with a bogus analogy to imply Chomsky is a kind of deluded religious fanatic. Then it concludes with a note complaining that people don't trust the objectivity of the author's opinions now that he's employed by the entity he's defending - well, yes, of course they don't, that's the compromise you made when you decided to take Open AI's money.

jacknews 4y ago

"For instance, a young child acquiring a language is developing — unconsciously, automatically and speedily from minuscule data — a grammar, a stupendously sophisticated system of logical principles and parameters. " LOL, this seems completely wrong to me. I think Chomsky's idea is that humans have some kind of 'grammar engine' that just needs configuring from 'miniscule data' aka many years of listening and practice speaking. No doubt humans have some optimized areas for language, but I doubt it's anything so formal.

dang 4y ago

Related ongoing thread: The False Promise of ChatGPT - https://news.ycombinator.com/item?id=35067619 https://news.ycombinator.com/item?id=35067619 - March 2023 (20 comments)

DiscourseFan 4y ago

Fundementally Chomsky has always argued that language must have some sort of core logic beyond Saussure's division between Signifier and Signified, but so far nobody has been able to prove it. What might be most disturbing to him is that LLMs, which operate purely on mass associations, might be closer to human cognition than a notion of cognitive logic. What if the associations are more primary, and the logic comes later? Just as someone else commented, animals work mostly in cycles, patterns, they don't have logic but the cycles of life are burned into them; there could be some magic key that explains why humans possess logic, but it would not be hidden in biology--more likely, since language is by nature a social phenomenon, it would be an aspect of culture.

lisasays 4y ago

In this piece Chomsky, the intellectual godfather of an effort that failed for 60 years to build machines that can converse in ordinary language. Woah there. When did Chomsky ever take an interest in building machines to do anything? What is he even referring to here? From the get-go, this piece sounds like an ad-hominem slam.

obblekk 4y ago

These are fair criticisms but imo miss Chomsky's much bigger point. From Chomsky's essay, 2 important lines: "Whereas humans are limited in the kinds of explanations we can rationally conjecture, machine learning systems can learn both that the earth is flat and that the earth is round. They trade merely in probabilities that change over time. For this reason, the predictions of machine learning systems will always be superficial and dubious." He's saying we trust humans because they can say things like "I'm pretty sure X because of explanation Y" and under the hood, we process the explanation, form our own probability of X and trust our own computation. But since LLMs cannot provide explanations for their beliefs, humans will never be able to rely on LLMs because the way we actually communicate is through explanations, not probabilities. Chomsky's conclusion: this is a good predictor, but not a human. Where Chomsky is actually wrong is he mixes up how good are you at prediction vs. how well can you convince a human of the prediction vs. how intelligent you are. We humans use a combination of accuracy + convincingness as a marker of intelligence. An AI that was just as accurate but 0 ability to convince a human because 0 ability to produce explanations could still be intelligent. This AI would seem like an alien to us, but an alien that could uncannily beat us at any challenge that requires an understanding of the natural world. In fact, we might never truly understand its internal explanations of the world, but still acknowledge it has them and is good at building new ones internally. Imagine being in a room with a foreign language speaker who beats you at chess. Clearly they have a mental model that works, even if they cannot explain it to you.

wg0 4y ago

Is this complete blog post or got trimmed? Because it ends with a claim about Galileo's telescope thus dismissing the critics via an analogy but doesn't offer much concrete chain of reasoning to reach there.

earfort 4y ago

[flagged]

theknocker 4y ago

[dead]

mikepalmer 4y ago

Regarding Chomsky's characterization of LLMs "gorging on hundreds of terabytes of data" compared to the "miniscule data" required for a child to learn a language: The brain is evolved, so the the "gorging" already happened in animals for 100s of millions of years. The brain has a lot of evolved sequence processing, visual processing, language (the authors are linguists and they admit this though it undercuts their point). Only fine tuning of this pretrained model is needed for a child to grow up speaking, say, English vs. French. This requires only a relatively miniscule amount of data. Moreover, it doesn't matter that LLMs work differently from the human brain. Per Larry Wall, TIMTOWTDI ("There is more than one way to do it").

morelisp 4y ago
> 100s of millions of years. Most animals don't acquire language, and the non-humans who are the best candidates for it all appear to have developed it independently; specifically in humans the potential is at most around 10mil years ago since we don't find it in other closely-related primates. Actual language is ~3 orders of magnitude younger, and during most of the time since then humans didn't have such high-density ways to ingest it. As I mentioned elsewhere in the comments, GPT-3 consists of around a million modern-human-years of linguistic intake. It seems to have the strong lead here. > it doesn't matter that LLMs work differently from the human brain. It does if your goal is to learn how human language acquisition works, rather than grind through another trillion in VC cash via your mechanical parrots.

hackandthink 4y ago

This Chomskyism and Chomsky hate is strange: "Yes, they’re wrong, and yes, despite being wrong they’re self-certain, hostile, and smug, and yes I can see this, and yes it angers me" Norvig's essay is less hateful and goes deeper. The Norvig - Chomsky Debate https://news.ycombinator.com/item?id=34857287 https://news.ycombinator.com/item?id=34857287