6 ms·

When Using AI they say "Context is King". "Reasoning" models are using the AI to generate context. They are not reasoning in the sense of logic, or philosophy

• bongodongobob 1y ago
  And yet it improves their problem solving ability.
• adastra22 1y ago
  You are making the same mistake OP is calling out. As far as I can tell “generating context” is exactly what human reasoning is too. Consider the phrase “let’s reason this out” where you then explore all options in detail, before pronouncing your judgement. Feels exactly like what the AI reasoner is doing.
  • stonemetal12 1y ago
    "let's reason this out" is about gathering all the facts you need, not just noting down random words that are related. The map is not the terrain, words are not facts.
    • deleted 1y ago
      [deleted]
    • energy123 1y ago
      Performance is proportional to the number of reasoning tokens. How to reconcile that with your opinion that they are "random words"?
      • kelipso 1y ago
        Technically random can have probabilities associated with them.. Casual speech, random means equal probabilities, or we don’t know the probabilities. But for LLM token output, it does estimate the probabilities.
        • energy123 1y ago
          Greedy decoding isn't random.
      • blargey 1y ago
        s/random/statistically-likely/g Reducing the distance of each statistical leap improves “performance” since you would avoid failure modes that are specific to the largest statistical leaps, but it doesn’t change the underlying mechanism. Reasoning models still “hallucinate” spectacularly even with “shorter” gaps.
        • ikari_pl 1y ago
          What's wrong with statistically likely? If I ask you what's 2+2, there's a single answer I consider much more likely than others. Sometimes, words are likely because they are grounded in ideas and facts they represent.
          • blargey 1y ago
            > Sometimes, words are likely because they are grounded in ideas and facts they represent. Yes, and other times they are not. I think the failure modes of a statistical model of a communicative model of thought are unintuitive enough without any added layers of anthropomorphization, so there remains some value in pointing it out.
    • CooCooCaCha 1y ago
      Reasoning is also about processing facts.
    • ThrowawayTestr 1y ago
      Have you read the chain of thought output from reasoning models? That's not what it does.
    • adastra22 1y ago
      So when you say “that’s the reason this out” you open up Wikipedia or reference textbooks and start gathering facts? I mean that’s great, but I certainly don’t. Most of the time “gathering facts” means recalling relevant info from memory. Which is roughly what the LLM is doing, no?
  • mdp2021 1y ago
    But a big point here becomes whether the generated "context" then receives proper processing.
    • adastra22 1y ago
      What processing? When you have an internal line of thought, what processing do you do on it? For me, it feels like I say something, and in saying it, and putting it into words, I have a feeling about whether it is true and supported or not. A qualitative gauge of its correctness. A lot of my reasoning is done this way, trusting that these feelings are based off of a lifetime experience of accumulated facts and the another things currently being considered. Explain to me how this is different than a neural net outputting a weight for the truthiness of the state space vector?
      • mdp2021 1y ago
        I meant the LLM. I meant, with «whether the generated "context" then receives proper processing», whether the CoT generated by the LLM and here framed as further "context", regardless here of how properly it is generated, receives adequate processing by the internals of the NN. A good context (any good context) does not necessarily lead to a good output in LLMs. (It does necessarily lead to a better output, but not necessarily a satisfying, proper, decent, consequential one.)
  • slashdave 1y ago
    Perhaps we can find some objective means to decide, rather than go with what "feels" correct
    • adastra22 1y ago
      That’s not how our brains work though, or how must examples of human reasoning play out. When asking “do LLMs reason” we are asking whether the action being performed is similar to regular humans, not some platonic ideal of a scientist/rationalist.
      • mdp2021 1y ago
        > When asking “do LLMs reason” we are asking whether the action being performed is similar to Very certainly not. We ask if the system achieves the goal. "When we ask if the coprocessor performs floating point arithmetic, we ask if the system achieves the goal (of getting accurate results)". Not, "does the co-processor ask if we have a spare napkin".
  • phailhaus 1y ago
    Feels like, but isn't. When you are reasoning things out, there is a brain with state that is actively modeling the problem. AI does no such thing, it produces text and then uses that text to condition the next text. If it isn't written, it does not exist. Put another way, LLMs are good at talking like they are thinking. That can get you pretty far, but it is not reasoning.
    • double0jimb0 1y ago
      So exactly what language/paradigm is this brain modeling the problem within?
      • phailhaus 1y ago
        We literally don't know. We don't understand how the brain stores concepts. It's not necessarily language: there are people that do not have an internal monologue, and yet they are still capable of higher level thinking.
        • chrisweekly 1y ago
          Rilke: "There is a depth of thought untouched by words, and deeper still a depth of formless feeling untouched by thought."
        • adastra22 1y ago
          I am one of those for what it’s worth. I struggle to put my thoughts into words, as it does not come naturally to me. When I think internally, I do not use language at all, a lot of of the time. Drive my wife crazy as my answer to her questions are always really slow and considered. I have the first think what thought I want to convey, and then think “how do I translate this into words?”
    • Enginerrrd 1y ago
      The transformer architecture absolutely keeps state information "in its head" so to speak as it produces the next word prediction, and uses that information in its compute. It's true that if it's not producing text, there is no thinking involved, but it is absolutely NOT clear that the attention block isn't holding state and modeling something as it works to produce text predictions. In fact, I can't think of a way to define it that would make that untrue... unless you mean that there isn't a system wherein something like attention is updating/computing and the model itself chooses when to make text predictions. That's by design, but what you're arguing doesn't really follow. Now, whether what the model is thinking about inside that attention block matches up exactly or completely with the text it's producing as generated context is probably at least a little dubious, and its unlikely to be a complete representation regardless.
  • kelipso 1y ago
    No, people make logical connections, make inferences, make sure all of it fits together without logical errors, etc.
    • pixl97 1y ago
      These people you're talking about must be rare online, as human communication is pretty rife with logical errors.
      • mdp2021 1y ago
        Since that November in which this technology boomed we have been much too often reading "people also drink from puddles", as if it were standard practice. That we implement skills, not deficiencies, is a basic concept that is getting to such a level of needed visibility it should probably be inserted in the guidelines. We implement skills, not deficiencies.
      • kelipso 1y ago
        You shouldn’t be basing your entire worldview around the lowest common denominator. All kinds of writers like blog writers, novelists, scriptwriters, technical writers, academics, poets, lawyers, philosophers, mathematicians, and even teenage fan fiction writers do what I said above routinely.
    • adastra22 1y ago
      How do they do that? Specifically, how? Moment by moment what does that look like? Usually it involves e.g. making a statement and “noticing” a contradiction in that statement. Very similar to how an LLM reasons. I think a lot of people here think people reason like a mathematical theorem prover, like some sort of platonic ideal rationalist. That’s not how real brains work though.
      • kelipso 1y ago
        Noticing a contradiction when making a statement is just one way writers find contradictions, and that kind of immediate noticing would only be for obvious contradictions. There are tons of other things you do, like recalling relevant facts related to the new statement and making sure the statement fits into the facts. You go through a timeline, make sure the statement fits into the timeline. You look at the implications of the statement, make sure those fit with other relevant facts. You opt to do these things depending on what the sentence means and implies. This is not just you "noticing" a contradiction, it's a process. And what does how real brains work mean anyway. You can't compare writers thinking and writing a novel to some six year old writing a paragraph.
  • viccis 1y ago
    >As far as I can tell “generating context” is exactly what human reasoning is too. This was the view of Hume (humans as bundles of experience who just collect information and make educated guesses for everything). Unfortunately, it leads to philosophical skepticism, in which you can't ground any knowledge absolutely, as it's all just justified by some knowledge you got from someone else, which also came from someone else, etc., and eventually you can't actually justify any knowledge that isn't directly a result of experience (the concept of "every effect has a cause" is a classic example). There have been plenty of epistemological responses to this viewpoint, with Kant's view, of humans doing a mix of "gathering context" (using our senses) but also applying universal categorical reasoning to schematize and understand / reason from the objects we sense, being the most well known. I feel like anyone talking about the epistemology of AI should spend some time reading the basics of all of the thought from the greatest thinkers on the subject in history...
    • js8 1y ago
      > I feel like anyone talking about the epistemology of AI should spend some time reading the basics I agree, I think the problem with AI is we don't know or haven't formalized enough what epistemology should AGI systems have. Instead, people are looking for shortcuts, feeding huge amount of data into the models, hoping it will self-organize into something that humans actually want.
      • viccis 1y ago
        It's partly driven by a hope that if you can model language well enough, you'll then have a model of knowledge. Logical positivism tried that with logical systems, which are much more precise languages of expressing facts, and it still fell on its face.
    • adastra22 1y ago
      FYI this posts comes off as incredibly pretentious. You think we haven’t read the same philosophy? This isn’t about epistemology. We are talking about psychology. What does your brain do when we “reason things out”? Not “can we know anything anyway?” Or “what is the correlation between the map and the territory?” Nor anything like that. Just “what is your brain doing when you think you are reasoning?” And “is what an LLM does comparable? Philosophy doesn’t have answers for questions of applied psychology.
• ofjcihen 1y ago
  It’s incredible to me that so many seem to have fallen for “humans are just LLMs bruh” argument but I think I’m beginning to understand the root of the issue. People who only “deeply” study technology only have that frame of reference to view the world so they make the mistake of assuming everything must work that way, including humans. If they had a wider frame of reference that included, for example, Early Childhood Development, they might have enough knowledge to think outside of this box and know just how ridiculous that argument is.
  • gond 1y ago
    That is an issue prevalent in the western world for the last 200 years, beginning possibly with the Industrial Revolution, probably earlier. That problem is reductionism, consequently applied down to the last level: discover the smallest element of every field of science, develop an understanding of all the parts from the smallest part upwards and develop, from the understanding of the parts, an understanding of the whole. Unfortunately, this approach does not yield understanding, it yields know-how.
    • Kim_Bruning 1y ago
      Taking things apart to see how they tick is called reduction, but (re)assembling the parts is emergence. When you reduce something to its components, you lose information on how the components work together. Emergence 'finds' that information back. Compare differentiation and integration, which lose and gain terms respectively. In some cases, I can imagine differentiating and integrating certain functions actually would even be a direct demonstration of reduction and emergence.
      • gond 1y ago
        Yeah that’s a nice addition. However, remember that reassembling is synthesis, not emergence. Emergence is what you /may/ get by reassembling, but must not. We are talking about systems, thus, in the end, you are correct. It’s just that the terms seem to be a bit muddled.
  • dmacfour 1y ago
• cyanydeez 1y ago
  They should call them Fuzzing models. They're just running through varioous iterations of the context until they hit a token that trips them out.
• benreesman 1y ago
  People will go to extremely great lengths to debate the appropriate analogy for how these things work, which is fun I guess but in a "get high with a buddy" sense at least to my taste. Some of how they work is well understood (a lot now, actually), some of the outcomes are still surprising. But we debate both the well understood parts and the surprising parts both with the wrong terminology borrowed from pretty dubious corners of pop cognitive science, and not with terminology appropriate to the new and different thing! It's nothing like a brain, it's a new different thing. Does it think or reason? Who knows pass the blunt. They do X performance on Y task according to Z eval, that's how you discuss ML model capability if you're persuing understanding rather than fundraising or clicks.
  • Vegenoid 1y ago
    While I largely agree with you, more abstract judgements must be made as the capabilities (and therefore tasks being completed) become increasingly general. Attempts to boil human intellectual capability down to "X performance on Y task according to Z eval" can be useful, but are famously incomplete and insufficient on their own for making good decisions about which humans (a.k.a. which general intelligences) are useful and how to utilize and improve them. Boiling down highly complex behavior into a small number of metrics loses a lot of detail. There is also the desire to discover why a model that outperforms others does so, so that the successful technique can be refined and applied elsewhere. This too usually requires more approaches than metric comparison.