11 ms·

> The large language models are a little surprising. I’ll give you that. I think this is the key point about LLMs that kind of explains the wide and polarized

• JoshTko 3y ago
  The more I think about it the more I'm convinced I am basically just predicting/saying my next word whenever I speak.
  • morkalork 3y ago
    Why not? Your brain is already making predictions about what you expect to see and hear as a part of your perception of reality anyways.
    • quad_eye_oh 3y ago
      Right, but LLMs suggest that learning how to predict and only training on predictions is sufficient to learn anything and to have emergent generative abilities. What if learning to predict the upcoming input is all that's all that is needed for general human intelligence? What if it is all that any animals do?
      • 20after4 3y ago
        This right here. I actually am strongly starting to believe that this is indeed what's going on. I read the book Kingdom of Speech a few years ago and that also left me with the perspective that perhaps language has a lot more to do with how we think and perceive the world than most people like to admit. The book has been heavily criticized but I believe it made an interesting point about language.
        • moffkalast 3y ago
          I mean hasn't it been proven on multiple occasions that humans who never learn a language during their life also don't develop higher cognitive skills and stay on the level of a child? Clearly language plays a notable part in reasoning, and it's frankly no surprise that we're seeing this behaviour from language models. It's also quite interesting how foundation models and fine tuning appear analogous to being born with a brain that already has six million years' worth of weights in it (trained implicitly through random changes and natural selection), which are then adapted over the course of a lifetime when environment relevant data is gradually obtained.
  • quad_eye_oh 3y ago
    This is a thought that LLMs caused me to have: the emotion of surprise is just the brain model of the world discovering that it had been poorly calibrated, or making poor predictions.
    • Hendrikto 3y ago
      Sounds like the free energy principle [0]: > The free energy principle is based on the Bayesian idea of the brain as an “inference engine.” Under the free energy principle, systems pursue paths of least surprise, or equivalently, minimize the difference between predictions based on their model of the world and their sense and associated perception. [0]: https://en.m.wikipedia.org/wiki/Free_energy_principle https://en.m.wikipedia.org/wiki/Free_energy_principle
  • mewse 3y ago
    The question is whether you knew how that sentence was going to end when you started writing it, or indeed whether I knew that I was going to add this comma-separated adjunct when I started writing the preceding clause, and I cannot honestly say at this precise moment of typing whether the final word in this sentence is going to end up being 'yes' or 'no'.
    • Mertax 3y ago
      I do wonder how well (quickly) the whole thought is formed in your head and it’s just the encoding into language that tricks you into thinking you didn’t know what the word would be. Having learned another language, the moment you start to feel “fluent” is when you start speaking first in the 2nd language and aren’t using your first language as an intermediate step to translate to your 2nd language.
    • diffxx 3y ago
      For me it's a little more subtle. When I sit down to write, say, an email, I usually have a specific point that I want to make but I usually haven't yet figured out exactly how I want to get there. I'm not even sure it is even possible to conceive of writing without doing it word by word. Perhaps the difference though is that we have the ability to go forward and backwards in the sense that we can usually think faster than we can type or write which allows our thoughts to advance beyond the tokens we have produced. This allows for a kind of real time editing process that incorporates both past and possible future information. But I imagine there us probably a way to enhance LLMs to branch and edit before producing the next word materially (if they aren't already doing that). To go even more meta, there is an analogy I'm trying to make right now in which I am visualizing a road and thinking about how describing the road relates to the process of writing. In my mind's eye, I can see the full length of the road and all of its contours but I can't actually describe the individual stretches of the road coherently without enumerating them. Something similar happens with writing. I can visualize what I want to say far beyond the next word, but it's true that the actual process of writing goes word to word, much like how the process of token selection is described for an LLM. The question is whether the LLM has an analogous conception of where it is going. Going back to the process above, sometimes I know where I am going and haven't yet figured out how to articulate it yet. It is through the process of writing that I am able to articulate that thought. But the thought preceded my articulation of it. I don't know to what extent LLMs have coherent thoughts that they are articulating or if that even makes sense for the type of intelligence they project. My suspicion is that they don't have additional sensory inputs beyond language that give thoughts the immaterial shape that then is expressed in language. Without that, I am skeptical that they will truly get beyond regurgitating and/or remixing what has already been fed to them textually. That doesn't diminish how amazing they are, but I am somewhat more in the Brooks/Knuth camp that they are impressive and surprising, but there is something that ultimately leaves me a bit cold about them.
  • cageface 3y ago
    Maybe in casual conversation but that's not how I experience my though process about anything non trivial at all. I usually spend a lot of time thinking about the concept in non verbal terms and that process involves recalling images and sensory information in fairly abstract terms and then through what feels like several iterations it starts to coalesce into something I can encode in language. I think we can all agree that these LLMs are surprisingly good at generating text that is often coherent but I don't see how you can discard all those extra inputs and claim you have the same process.
    • panarky 3y ago
      > thinking about the concept in non verbal ... abstract terms The abstract terms we think about are concepts, and we think about multiple concepts, at various levels of abstraction, and their relationships to each other, before getting a sense of what we want to say or write. Only then do we begin speaking or writing, grouping concepts into paragraphs, breaking them down into sentences and words. And there's evidence that LLMs do something similar, creating embeddings for both big ideas and small details, modeling how the small details combine into larger concepts, discovering the relationships between concepts, and only then generating a probabilistic sequence of tokens to express those deeper concepts.
      • cageface 3y ago
        Right but they're limited to concepts that are derived from linguistic tokens. Human reasoning can include nonverbal inputs. For example asking GPT what would be a reasonable way to stack a list of arbitrary objects on top of each other.
        • namaria 3y ago
          Lately I've been think that staple of science fiction, the android robot, is the only way to get AGI. I thought a lot about computer vision when that was the bleeding edge of AI a few years back. It strikes me as obviously wrong to try training models on two dimensional arrays of pixels and expect them to develop visual skills. Why don't we give them two cameras and arms so they can learn about physics and start differentiating 3 dimensional input? I think when we have robots capable of interacting with the world as humans do we might be able to train them to develop near human levels of intelligence.
  • astrea 3y ago
    But you and your brain are much more than the language-forming aspect.
    • moffkalast 3y ago
      Yes, most of the brain is a glorified microcontroller for running and interpreting data from a billion skin sensors and driving hundreds of muscle actuators, doing depth and object inference from binocular images, locating sounds from the twin microphone array, etc. Language and especially reasoning is a very small part of it all.
    • JoshTko 3y ago
      Language is the compression of our entire human experience into words, so I'd argue language captures the majority of the thinking aspect of the human experience.
  • ChatGTP 3y ago
    I believe this view of humans is because we work in increasingly deterministic environments (the internet) using predictable methods of communication, text. We’ve kind of made the world a bit boring and deterministic, places where almost perfect is knowledge is obtainable and so everything feels more and more predictable. Your day probably consists of using Google, talking on platforms that don’t change much, solving already solved coding problems and communicating with others about office politics problems that we’ve all spoke about over and over again. We literally are just chatbots in this world.
  • zmmmmm 3y ago
    Honestly I feel like one reason people are struggling with this is they can't accept a critical part of the truth : most people are stochastic parrots themselves most of the time. True, creative genuine deep thinking is an exceptional state of thinking for us.
    • DJHenk 3y ago
      Years ago I transitioned from a developer role to a manager role and suddenly I had to do a lot more talking. Not all the talking needs to be a deeply involved exchange of complex ideas, a lot of it serves a different purpose. Sometimes it can be a simple as filling up the time in a pleasant way with a group of people that may or may not know each other that well. After getting some experience with this I noticed that I had developed a talking on/off button in my head. I could just simply turn it on and start talking. I could generate words that sounded good together and fit the purpose of the moment. But They just seemed to come from a different place in my brain than my conscious mind. Because that was not involved in this process at all. The only job my mind had was to turn the button off again at the right moment, for the rest it was free to think whatever it wanted. (I transferred back to development a couple of years later.)
      • namaria 3y ago
        The very fact that we can be conscious about our ability to generate language and work it separately from the rest of our consciousness tells me that chatbots might emulate our language generation ability. We shouldn't invert it to conclude that means they can reason about it.
        • TeMPOraL 3y ago
          > We shouldn't invert it to conclude that means they can reason about it. Agreed. However, I think it's somewhat accepted view that the bulk of work constituting reasoning happens subconsciously, with the conscious mind playing the role of a censor/gatekeeper, and occasionally handholding during when reasoning through tougher problems.
    • 3y ago
  • anon84873628 3y ago
    The sad truth is that we are all NPCs without free will. "Thoughts" are a non-physical phenomenon that, seemingly, arise from physical activity in the brain. The thoughts (our "experience") come after the electrochemical reaction that manifests them. How could these non-physical phenomena then turn around and influence the physical chemistry? They can't. By the time we experience a thought, the physical state for it has been represented, and that state causes the next one after it. Like a stone bouncing down a hill - chaotic but still deterministic. We're just along for the ride.
    • antihipocrat 3y ago
      The act of being conscious of our own thoughts and using them to influence our physical state is skill practiced by many people with success.
      • anon84873628 3y ago
        Nope, that's just what it feels like we're doing. It's a good trick!
    • namaria 3y ago
      Reducing a debate that has been raging for millennia across several disciplines to three lines of text tells me you don't have enough of a grasp on the subject to draw any conclusions about it, let alone resolve it altogether.
      • Biganon 3y ago
        Care to enlighten us a little? What is there even to debate? The position expressed by your parent comment is so uncomfortable that it's constantly being challenged, but that doesn't make it wrong. Or are "thoughts" some sort of magical fluff not bound by the laws of physics? Are we constantly pulling ourselves by our bootstraps? Giving birth to entirely new brain activity out of nothingness? This is just woo woo nonsense. Until someone can find an explanation that makes more sense than "natural selection slowly gave apes the ability to think abstractly and they started thinking about themselves", I won't believe in any kind of free will, and now the meta is reaching the point where we create entirely new brains out of silicon. "They just repeat stuff!" So do we
  • namaria 3y ago
    Can you think without words? Can you imagine how it feels to swim, something you might have done, and extend it to fly like a bird? Can you read a novel and feel emotions about characters that don't exist beyond text descriptions of their actions? Language is a way to encode thought. It's not what thoughts are made of.
    • copperx 3y ago
      > Language is a way to encode thought. It's not what thoughts are made of. What about people who think verbally and don't have a "mind's eye"? In that case, thoughts and their encoding might be closer to 1:1.
      • namaria 3y ago
        Are images and words the only way to think? Are these boundaries even fundamental or are they arbitrary? How does a blind deaf person think? Do you ever think in terms of sensations or movements? Can you remember a roller coaster ride, a car trip? Can you think of a taste? I find hard to conceive of people who can think only in terms of language.
  • goatlover 3y ago
    Does that reasoning work when applied to what you just wrote? If you’re just predicting the next word, then what makes you confident in the conclusion that you’re just predicting your next word?
    • JoshTko 3y ago
      Yes. This was my second draft. I reviewed my first draft which became an input in my second draft which gained more clarify after reviewing the first draft.
      • goatlover 3y ago
        The way people usually say it is that you revised your first draft upon reflection, because it didn't quite get across the point you wanted to make, or something to that effect. So what makes the way you state it more true than the way people typically state it?
        • JoshTko 3y ago
          I can switch from word generation and evaluation. I saw the score of 1st draft was below my target. And I used the insights from the eval of the 1st draft to inform the 2nd draft.
• mamp 3y ago
  It turns out to accurately predict the next word requires huge amounts of implicit contextual knowledge i.e. understanding about the world. Next word prediction was the trigger, the optimisation for the task results in a broad (but currently unreliable) model of the world.
• ramraj07 3y ago
  The issue with all these experts is they still think it's human nature to be able to fully understand the world before they speak about it. On the contrary it's human nature (and all animal nature) to figure out how to navigate the world without fully understanding or having a complete model of it. All you need is a working model that affects the facets of the world you need to deal with. I still remember in the 90s my school friend came over to my house and I was sending a fax for my dad. He was surprised the paper came back out the other side. He wasn't an idiot, and he was 15. But his model of the world didn't include deep thought about how fax works, he just merely concoted a system where the paper just went through the wire. That moment stays with me and reminds me what it is to be human and think like one. I think chatgpt is like my friend, and that should scare and excite us.
  • zht 3y ago
    Now imagine if grown ass adults listened to your friend assuming he understood the world fully
    • famouswaffles 3y ago
      Nobody understands the world fully so that's cool. We don't even get sense data that isn't distorted by the brain.
  • _ea1k 3y ago
    I remember a friend being confused by downloading. He didn't realize that meant we'd have a copy and not that the source would lose it in the process. It seemed bizarre at the time, but, tbh, I didn't have _that_ much better of a model of how the whole process worked.
    • groestl 3y ago
      > we'd have a copy and not that the source would lose it in the process I don't known when this happened, but maybe that means the Anti-Piracy campaigns worked on him, confusing illegal replication with theft.
• consilient 3y ago
  > My current (tentative) resolution of the surprise is that language encoded way more information about reality than we thought it did. (Enough information that you can fully derive reality from language seems improbable, but iirc it did derive Othello and partly derived chess and I would have thought there wasn’t enough information in language to derive those without playing the games as well, so I can’t rule it out.) The information isn't in language itself, it's in language as actually used by humans. GPT4 knows about chess because it's "read" a significant fraction of everything we've ever written about chess. A human being who did that without ever playing a game would also start out better than a typical novice.
  • fwlr 3y ago
    > A human being who did that without ever playing a game would also start out better than a typical novice. I am quite skeptical of these arguments along the lines of “imagine a human read everything written on the topic…”. What humans are doing when they read something is not what neural nets are doing when they read something. Humans are (idealistically) doing something like Feynman’s description of how he reads (or in this case, listens to) a theorem: “I had a scheme, which I still use today when somebody is explaining something that I’m trying to understand: I keep making up examples. For instance, the mathematicians would come in with a terrific theorem, and they’re all excited. As they’re telling me the conditions of the theorem, I construct something which fits all the conditions. You know, you have a set (one ball) – disjoint (two balls). Then the balls turn colors, grow hairs, or whatever, in my head as they put more conditions on. Finally they state the theorem, which is some dumb thing about the ball which isn’t true for my hairy green ball thing, so I say, ‘False!’"” Bret Victor’s description of what “really good programmers” are doing is also related: “[showing the code for binary search] In order to write code like this, you have to imagine an array in your head, and you essentially have to ‘play computer’. You have to simulate in your head what each line of code would do on a computer. And to a large extent those who we consider to be skilled software engineers are just those people who are really good at playing computer.” I think when we imagine an LLM as a human who’s read everything ever written in chess but never played an actual game, we’re actually tricking ourselves - because that hypothetical human would be ‘playing chess’ inside their head by imagining the pieces and moving them according to the rules they had read[1]. LLMs are not doing anything like that when they read about chess. So it’s a very restricted (or perhaps more accurately, a very different) kind of ‘reading’ that we don’t have any intuition for. Since the ‘reading’ that we do have an intuition for is smuggling in exactly the kind of “modeling the world” ability we’re looking for, it’s not surprising that this argument would incorrectly lead us to believe we’ve found it in LLMs. 1: In fact the very best computer chess is achieved by AlphaZero which was trained exclusively on “playing chess in its head”, and it beats even the most powerful and optimized search algorithms like Stockfish looking 20 moves ahead.
• deleted 3y ago
  [deleted]
• causality0 3y ago
  If LLMs are an understanding of the world it would mean humans in a few decades found a way to create sapience with many orders of magnitude fewer interacting elements than evolution did. I find that doubtful, at least in light of the fact every other way we've replicated biological computation requires many more computational elements.
  • XorNot 3y ago
    Why would that be surprising though? The natural world climbs up to understanding from a severe disadvantage. Chemicals don't understand the world - the natural world had to be randomly build every bit of its systems from scratch, before it could produce a succinct descriptive system for it. LLMs on the other hand start out with a succinct descriptive system, and translate that to the world of chemicals and photons via some very complicated naturally evolved systems.
  • seanhunter 3y ago
    Firstly, if humans created sapience, we created it with all the interacting elements that evolution used plus all the things that we used. Secondly, maybe sapience isn't as big of a deal as we thought compared with all the other things that evolution did. Remember that biological entities have to figure out survival, reproduction etc. Sapience emerges as a byproduct but the selective pressure is towards those things so sapience is only selected for to the extent that it also moves forward those other goals. By contrast, LLM training is just focussed on the task of making the model better. The model doesn't have to figure out how to feed itself, ward off predators, not accidentally die in the myriad ways things die, reproduce itself etc. It's way more specific. It doesn't seem unreasonable to think that the complexity would be lower given it's not trying to achieve nearly as much.
    • furyofantares 3y ago
      Alright, I give. What the heck is "sapience"?
      • seanhunter 3y ago
• astrange 3y ago
  I think the main issue is that since we don't have access to the implementation of GPT-4, there's no way for researchers to know how it works, so either they're not saying anything or they're just making things up. GPT-4 is "multimodal" and RLHF'd, so it was trained with some tasks other than next word prediction. I don't remember if it's been trained for code correctness (by running unit tests etc.), but other models have been.
  • coldtea 3y ago
    That's not much of a problem, as it only affects the "GPT" LLM. The same researchers can always study comparable in behavior LLMs like Llama and its descendants.
• fcanesin 3y ago
  We known mathematics is the science of reality patterns, maybe if we can keep LLMs current ability in natural language and keep improving its (current poor) math abilities it will get there?...
• klipt 3y ago
  > language encoded way more information about reality than we thought it did Language is roughly what separates humans from other apes ... so why would it surprise us that it encodes much of the information of civilization?
  • krainboltgreene 3y ago
    > so why would it surprise us that it encodes much of the information of civilization? The Humanities are largely considered superfluous in the tech world, why are you surprised that they are surprised?
    • klipt 3y ago
      To clarify, I'm including stuff like Python code under language, because these large language models are also trained on Python code.
      • krainboltgreene 3y ago
        Very very few people in tech are even experts in the meta analysis of programming languages, much less the importance of languages.
  • fwlr 3y ago
    I’m not as surprised as many people (I saw a great tweet once that said “Language was the first artificial intelligence, writing was the second. I literally believe this”, and that’s a broadly accurate description of my worldview too). That said, maybe some of the surprise comes from believing “the map is not the territory” and related ideas? We generally believe that the map is not the territory and this gives us some obviously correct intuitions (like “changing the map doesn’t change the territory”), but maybe it has also given us some subtly incorrect intuitions. I’m not talking about obviously incorrect, like “you can’t understand the territory just by looking at enough maps”. I mean something more subtly wrong. One candidate off the top of my head is an intuition that “maps approximate the territory but necessarily at a lower level of detail (a 1:1 map of the territory would be the same size as the territory), so your understanding of the territory can improve as you read more maps but it can’t improve on the limit of the most detailed map available, because that information literally isn’t there”. I could see that possibly being wrong somehow.
• thomastjeffery 3y ago
  > My current (tentative) resolution of the surprise is that language encoded way more information about reality than we thought it did. I think you are close to the mark, but you have been subtly mislead: language is not the data we are working with. We are working with text. Once you fix that particular failure of word choice, everything else becomes much more clear: text contains much more information than language. We aren't dealing with just any text, either: that would be noise. We're training LLMs on written text. Natural language is infamous for one specific feature: ambiguity. There are many possible ways to write something, but we can only write one. We must choose: in doing so, we record the choice itself, and all of the entropy that informed it. That entropy is the secret sauce: the extra data that LLMs are sometimes able to model. We don't see it, because we read language, not text. The big surprise is that LLMs aren't able to write language: they can only write text. They don't get tripped up reading ambiguity, but they can't avoid writing it, either. Who chooses what an LLM writes? Is it a mystery character who lives in a black box, or a continuation of the entropy that was encoded into the text that LLM was trained on?
  • fwlr 3y ago
    The idea that text contains more information than language is fascinating, wow. Thank you!
  • blowski 3y ago
    I don't really understand your distinction between language and text, but it sounds intrguing. Would you be able to give more detail? I searched but couldn't find anything that seemed to explain it.
    • j16sdiz 3y ago
      Spoken language have pitch, stress, mood, etc.. Written text contain some, but not all. Text, on the other hand, can present in list or table, with varies formatting, indentation. You can't reproduce them in spoken text.
    • bruce511 3y ago
      Not the poster, but for me it comes down to a mix of clarity and permanence. I teach both verbally (interactive question/answer) and I've also written text books. Verbally by language is "loose". I'll say class when I mean object, unicode when I mean utf-8 and so on. Sentences are not all well formed, and sometimes change mid-thought. It's very "real time" Writing is a lot more deliberate. I have to be sure of each fact I state. I often re-test things I'm only 95% sure about. I edit, restructure, remove, add, until I'm happy. Of course all communication falls on a spectrum. Think phone call at one end, text book on the other. When I do a verbal lecture I'm usually careful with my speech, and when I post on hacker-news less rigorous. Language covers all of it. Text skews to the more deliberate side. Cunningly the language models are trained using (mostly) text, not speech. That will have an impact on them.
• aappleby 3y ago
  What exactly does "understanding the world" really mean?
  • espe 3y ago
    compression :)
  • deleted 3y ago
    [deleted]
• winddude 3y ago
  > One way to resolve this surprise is to find some reason to believe these strange abilities are fundamentally not an understanding of the world. Thus stochastic parrots, this article, Yan LeCun and Chomsky, etc. This one. This is the correct resolution.
• dragonwriter 3y ago
  > One way to resolve this surprise is to find some reason to believe these strange abilities are fundamentally not an understanding of the world. Thus stochastic parrots, this article, Yan LeCun and Chomsky, etc. I mean, it is kind of obvious that predicting text based on a large corpus of text written by people with a model of the world will, if it works at all, look like having a model of the world. The question is whether that involves having a model of the world, and secondarily, if there is a difference, what is the practical impact of the difference? Or maybe that’s not really the question, because the whole “model of the world” thing is metaphysical omphaloskepsis that is inherently unanswerable because we can’t actually unambiguously reduce it to something with testable empirical predictions, reflecting a rationalization for elements of our culture and particularly our own view of our own special place in the universe, and the different answers to it likewise have no real meaning but simply reflect people’s bias for whether or not they want to share that special place, either in general or with the particular computing systems under discussion in particular.
  • tornato7 3y ago
    The one thing that has convinced me that chatGPT has built a real world model is asking it how to stack different objects. You can ask it for the safest way to stack a pillow, an egg, a nail, a book and an action figure. Even get more complicated. GPT-4 will, most of the time, correctly reason through how to stack these items to form a stable structure without breaking. That tells me it knows intimately the physical properties of all of these objects and how they interact with each other.
    • avereveard 3y ago
      Or was that one of the training samples provided to gpt 3.5? It is going to be increasingly harder to divide between what it was in the training set and what is creative thinking going forward. Like, at the very minimum, all the publicly known reasoning samples that were used to probe gpt version 3.5 downward are going to be unreliable for knowing what gpt4 reasoning abilities are
      • 3y ago
• yblu 3y ago
  What I found disheartening was many of those scientists, especially those on the "nothing to worry about" camp, seemed not to entertain the thought that they could be wrong, considering the scale of the matter, i.e. human extinction. If there's a chance AI poses an existential threat to us, even if it is 0.00000001% (I made that up), should they be at least a bit more humble? This is uncharted domain and I find it incredible that many talk like they already have all the answers.
  • akiselev 3y ago
    Meh. Add it to the pile. The number of world ending risks that we could be worried about at this point are piling up and AI exterminating us is far from the top concern, especially when AI may be critical to solving many of the other problems that are. Wrong about nuclear proliferation and MAD game theory? Human extinction. Wrong about plasticizers and other endocrine disruptors, leading to a Children of Men scenario? Human extinction. Wrong about the risk of asteroid impact? Human extinction. Climate change? Human extinction. Gain of function zombie virus? Human extinction. Malignant AGI? ehh... whatever, we get it. It's like the risk of driving: yeah it's one of the leading causes of death but what are we going to do, stay inside our suburban bubbles all our lives, too afraid to cross a stroad? Except with AI this is all still completely theoretical.
    • onethought 3y ago
      Yes, and all of those (including AI) are not even human extinction events. - Nuclear war: Northern Hemisphere is pretty fucked. But life goes one elsewhere. - Plasticisers: We have enough science to pretty much do what we like with fertility these days. So it's catastrophic but not extinction. - Climate Change: Life gets hard, but we can build livable habitats in space... pretty sure we can manage a harsh earth climate. Not extinction. - Deadly virus: Wouldn't be the first time, and we're still here. - Astroid impact: Again, ALL human life globally? Some how birds survived the meteor that killed the dinosaurs, I'm sure we'd find a way. - Complete Made up evil AI: Well we'd torch the sky, be turned into batteries but then be freed by Keanu Reeves.. or a Time traveling John Connor. (sounds like I'm being ridiculous, but ask a stupid question...)
• dirkt 3y ago
  > One way to resolve this surprise is to find some reason to believe these strange abilities are fundamentally not an understanding of the world. An easy way to test this is to ask questions and followup-questions that actually require understanding, and compare this to the answers. I recommend to try that.
• notShabu 3y ago
  Has there been any research in the possibility that much of what we say/speak (including this convo) is actually just stochastic parroting? e.g. instead of the stochastic parrots mimicking intelligence maybe intelligence doesn't exist, it's just stochastic parrots of various levels of sophistications organized into a hierarchy. "Intelligence" is necessarily socially defined with the more complex parrots being unpredictable and "intelligent" from the POV of lower parrots. Vicer versa, looking down, the lower parrots seem to act like "NPCs"
  • Sai_ 3y ago
    This is an interesting question. To paraphrase as per my understanding of your comment, is intelligence an emergent property of being able to interact with each other through language? Say I speak gibberish (to you) which is actually me explaining to you the theory of relativity, would you consider me intelligent?
    • anon84873628 3y ago
      What's the difference between "understanding" and "having really good probabilistic information about how words combine"? Kids learn to speak by parroting what they hear and observing the outcome. Then they run tests that reinforce the connections between words. That's what the model is. But humans also get to link words with all the other sense experience we have (like how sweet cherries, loud fire trucks, and that one crayon are all "red"). LLMs don't have as many dimensions of experience they can link to. But anyway, intelligence is about having an internal model of the world and using it to predict the future. The more rich and accurate the model, the more intelligent. The ability to communicate isn't a prerequisite; lots of animals have intelligence that isn't built with language.
      • namaria 3y ago
        Spot on. That we have internal language models doesn't mean we only have language models as cognition. People seem to be redefining imposter syndrome as reflected by stand alone language models. Makes me think of Tropic Thunder. "Am I even real". Lol get a grip people.
• nopinsight 3y ago
  There is in fact compelling evidence that language models can learn some world models. Here's a research paper on this and a blog post by the lead author summarizing the results. Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task. https://arxiv.org/abs/2210.13382 https://arxiv.org/abs/2210.13382 Do Large Language Models learn world models or just surface statistics? https://thegradient.pub/othello/ https://thegradient.pub/othello/ ----- An argument based on common sense can also be made: Any system that possesses a wide range of capabilities, most of which it was not specifically trained to perform, cannot possibly perform all these tasks so well solely by making probabilistic guesses. (Humans, too, were not directly shaped by natural and sexual selection to possess all of our cognitive capacities.)
  • whimsicalism 3y ago
    Beyond compelling evidence, it has been our working theory for how neural networks generalize for many years now.
• keyle 3y ago
  > or just put a few industries out of a job. Yeah no big deal. Happens in history all the time, within a year.
• whimsicalism 3y ago
  > But almost everybody is really surprised that becoming better at a task like ‘natural language prediction’ would produce all these strange abilities that sort of look like “understanding the world”. It honestly should not have been a surprise to anyone in the field at least in the last 6 years.
• DiogenesKynikos 3y ago
  I don't think it's at all surprising that a system designed to predict the next word ends up understanding the world. If you want to predict the next word accurately, you first have to know which words exist. To progress, you'll have to learn about the mechanics of grammar and which words are used more frequently or in combination. To become even more accurate, it helps to understand context, so that the sentences you string together will at least be relevant to the subject. If you want to increase your accuracy even further, you'll have to start memorizing all sorts of facts (e.g., "Who was the monarch of England in 1600?"). Being able to synthesize those facts into a coherent argument will increase your accuracy even further. In the end, predicting the next word accurately requires an understanding of the world. This isn't all that different from how our own intelligence evolved. You could look at humans from the outside and disparagingly point out that the ultimate purpose of the human brain is to direct muscle motions in a way that maximizes the chances of reproductive success. It just turns out that solving that problem effectively has led to the development of an enormously complicated piece of machinery, capable of synthesizing all sorts of input stimuli into a coherent picture of the world, and ultimately of producing the works of Shakespeare and the music of Beethoven.
• blazespin 3y ago
  The resolution is actually fairly simple. It's an incredibly brilliant stochastic parrot with some limited reasoning capabilities. Some folks will try to say it cannot reason, but they are wrong, there is extensive proof of that. The only question is how limited are its reasoning capabilities. After spending extensive time on openai/evals, having submitted 3 of my own, and doing a lot of tests, I would argue that an average person of average IQ could out think GPT4 - as long as the stochastic parrot aspect wasn't a factor.
  • sgt101 3y ago
    It's very hard to know because we don't know and can't experiment with its training data. So - it may be doing first principle reasoning, or it may be doing token substitution vs. some known example that it's seen before and is matching to.
    • blazespin 3y ago
      The dictionary definition of reasoning says nothing about how the thinking is done, only that it's sensible and logical, which is exactly what GPT4 is. Limited, yes, but it reasons.
    • moffkalast 3y ago
      > it may be doing token substitution vs. some known example that it's seen before and is matching to Ah, just like us with literally 99% of stuff taught in school you mean.
  • moffkalast 3y ago
    That's probably an accurate assessment, the question is mainly if the reasoning can be improved to a notable extent and how much on the current architecture. I myself assumed that we're pretty close to the end of the S curve when first using 3.5-turbo and figured that hallucinations will be pretty hard to overcome, but with GPT 4 being such a massive improvement on all metrics I'm no longer as sure. GPT 5 will probably be more definitive on what's possible, based on where it starts having diminishing returns.
• namaria 3y ago
  > My current (tentative) resolution of the surprise is that language encoded way more information about reality than we thought it did. "A whole mythology is deposited in our language" - Wittgenstein
• ilaksh 3y ago
  GPT-4 has also been trained on images.