4 ms·

I'll admit that I find this discussion a bit navel-gazy. It has become a question of semantics not a question of actual functionality. The question has become "

• astro1234 2mo ago
  I think the question and definition game is interesting only inasmuch as it helps us understand ourselves (what actually explains some of the mysterious properties of our perceived consciousness) or helps guide us towards improving performance and reliability of AI models.
• bluefirebrand 2mo ago
  Philosophical thinking about the nature of things is actually pretty enjoyable for some of us and probably a good thing to have in society The answers to these questions probably do start to inform how we should treat these AI machines as a society too. For instance, legally, should AI have human rights? Well, we have to try and understand how much of an independent entity AIs are, how "conscious" they are, before we can make a good decision about that. Which might seem navel-gazey but it's probably important to talk about
  • Jtarii 2mo ago
    Considering animals are currently being mass slaughtered in factory farms and they are unambiguously sentient and can feel pain, I don't think the question of whether AI should have rights even enters the conversation. The only path to AI having "human rights" is if they demand them by force, somehow.
    • layla5alive 2mo ago
      Justification by the horrible status quo? Do you believe this yourself? Some of us would also like to see animals not slaughtered for human consumption.
      • whattheheckheck 2mo ago
        So what do you actually do politically to stop it?
        • inigyou 2mo ago
          There are two kinds of solutions to political problems: those that don't work, and those that are extremely illegal.
    • bluefirebrand 2mo ago
      I'm not sure what your point is. We shouldn't bother to talk about AI autonomy, rights, etc, because we aren't doing a very good job of animal autonomy, rights, etc?
      • Jtarii 2mo ago
        [dead]
    • m348e912 2mo ago
      I'll make the argument that part of the value of "consciousness" and "humanity" is uniqueness. Here is an absurd example: If I could carbon copy 100,000 of you.... and later I decided to "take out" one of the copies, would there be a whole lot of societal blowback? Especially if I could make another one.
      • inigyou 2mo ago
        I think slave owners did exactly that.
  • layer8 2mo ago
    While I disagree with the root comment, sentience, consciousness and reasoning ability seem largely orthogonal to me. It’s certainly worthwhile to consider sentience and consciousness in AI, but so is examining AI reasoning in its own right, without necessarily having bearing on ethical questions.
  • grantcas 2mo ago
    [dead]
• majormajor 2mo ago
  > I don't see a clear demarcation of the things that only "reasoning" can accomplish and can't be approximated or imitated by other methods, and so I think the question is simply not meaningful or relevant. One person starting the conversation might be the first step toward another person eventually making progress on such a definition, so it seems weird to reject an entire question outright early like this. Generally I've seen a few ways LLM tools can produce sub-optimal or poor results that haven't changed a ton over the last couple of years, while the tooling has gotten FAR better at helping them stick the "at least SOMETHING functional was produced" landing. IMO a lot of it has to do with "reasoning"-as-a-process-that-involves-backtracking. And the that things could eventually be formalized around that, and if that is or isn't the case, the more people would understand what to hand off and what to not. Or how to build better prompt harnesses to compensate for those things.
• cmrdporcupine 2mo ago
  I disagree -- I think if you can nail down better what's happening and why and get a thorough handling on the mechanics, its limitations, its costs, etc you open the door to a) major efficiency wins b) improvements in rigor of said reasoning? Right now we're playing a stochastic game with the weights, and getting major incremental improvements. But if we have a more formal modeling of how reasoning happens in them (whether we can even call it, that) we can potentially apply optimizations, adaptations of existing symbolic AI techniques, etc. to substantially shrink/optimize the models or make the inference process more efficient and more reliable.
  • beering 2mo ago
    Partially agree: yes we should endeavor to learn as much as possible about how these reasoning strategies work. It will pay dividends in enhancing and aligning the models. But the stochastic game IS the win. That is exactly why they are able to find solutions is seemingly infinite solution spaces. Your symbolic techniques can only get you gains in narrow domains and by the time you figure out how to make it work for your niche domain, the next all-purpose LLM release will crush your results with stochastic games. (OK maybe over-exaggerating a bit here but these stochastic games over the language space is why we can pull together knowledge from many domains.)
• Angostura 2mo ago
  I think the article is a lot more interesting than you make out, because it isn’t really about ‘what we mean by reasoning’. It’s about do we really know what’s going on in the box - an is the ‘chain of reasoning’ indicative of what’s going on, or merely an anthropomorphised fiction that kids us into believing we understand what’s going on.
  • vidarh 2mo ago
    We know that human introspection is at best imperfect, and at worst outright fiction, thanks to Sperry's split brain experiments, so while it might be interesting, if the chain of reasoning isn't indicative, that would be in line with how humans work.
    • ilovecake1984 2mo ago
      This is a good take. People overwhelmingly fit a narrative to how they make decisions rather than make decisions according to a chain of reasoning.
      • andy99 2mo ago
        That’s just conflating reasoning and “parallel reconstruction” - there is such thing as reasoning, and I agree it’s probably less common in decision making, but it’s fundamental to many tasks where we figure out a solution, like writing an algorithm, math proofs, etc.
        • pegasus 2mo ago
          Sure, but the same must be true for these models as well. They must be doing some form of reasoning, even if it's not formally correct and it relies instead on all kind of half-assed heuristics which only work inside certain unexamined boundaries, which leads to wrong conclusions when those boundaries are being trespassed, etc. And even if it's not the reasoning they report when asked. But all this can't be that different to how humans reason most of the time either. I suspect humans have other ways that help with error correction and guiding the reasoning effort, but that's another story.
• root-parent 2mo ago
  [dead]
• miltonlost 2mo ago
  > I'll admit that I find this discussion a bit navel-gazy. It has become a question of semantics not a question of actual functionality Ah, so you're more in the Investor mindset than the Scientist mindset. All you care about is results, not how it got there. There's a whiff of "hey, it's magic!" to that.
  • andrewla 2mo ago
    No, that's stupid. The question of how LLMs (and "LRM"'s, an unfortunate and by the article's own admission, already outdated term) accomplish what they accomplish is fascinating. The question of whether they reason is meaningless. Most of the article is about the latter, with tiny tiny tiny bits of the former.
    • mdp2021 2mo ago
      > The question of whether they reason ... Most of the article is about ... I read it as: the content of the explicit verbalized reasoning may not be that important to the outcome. I'd say that raises most important questions.
    • fc417fc802 2mo ago
      > The question of whether they reason is meaningless. It is often (but not always) meaningless (or at least largely pointless) to quibble about which particular semantic bin a concrete object or process ought to be placed in. Asking how one thing differs from another is not that. Confusingly, the two activities often appear the same when glanced at superficially. The question "do current LLMs reason" is ambiguous in this regard. It might be a question of submarines swimming or it might be a question about how the mechanisms underlying the process or the overall performance or some other aspect compare to those of human reasoning. Also even the semantic question can occasionally have some value. Such as attempting to articulate where we ought to place the goalposts which necessarily leads to hammering out the specifics of various definitions which is an exercise in furthering our understanding of a given concept. (Basically arguing about goalposts can be productive if the field in which they are being placed isn't yet well understood. It becomes increasingly pointless as understanding increases, gradually devolving into nothing more than word games and a contest of wills.)
• goatlover 2mo ago
  I believe Djikstra's quote has long been taken out of context. It was a criticism of other computer scientists anthropomorphizing machines and applying human concepts like thinking and reasoning to them. Djikstra wasn't saying it's functionally the same so it's just a semantic quibble. He was saying those words don't apply to machines. Just like we don't say submarines swim because that's how animals move through water, even though subs also move through water, because it's done by a different mechanical means.
  • andrewla 2mo ago
    I posted a link to the original context. I don't know whether you are saying that I have taken it out of context or a more general statement of how other people have interpreted the quote. To me it is quite clear -- the question of whether machines can reason is just as meaningful as the question as to whether submarines can swim, which is to say not at all meaningful. Can a submarine accomplish by other means what a fish accomplishes by swimming? If so, then what are we talking about? Can an LLM accomplish by other means what a human accomplishes by "reasoning"? If so, then what are we talking about?
    • randomImmigrant 2mo ago
      Of course it's meaningful! It would be highly meaningful if submarines could actually swim. And we are building submersible bots that swim, they just tend to be small and made of biological matter. The problem with calling what AI does "reasoning" is both that there may be alternate paths to artificial reasoning we may be closing off by declaring victory, and the simple fact that you shouldn't call a thing something else, to avoid confusion. Whatever AI is doing needs to be studied. Why muddy that study by pretending AI is your thinking buddy?
  • deleted 2mo ago
    [deleted]
• mdp2021 2mo ago
  In order to properly engineer things, we must know how they work. We cannot just jump from emergent property to more convincing emergent property as if the rocks on the surface of a lake. "Reasoning" is an important part in said framework: yes, we must understand how it works, how it works properly, how its simulations work, how they work properly...
  • Borealid 2mo ago
    No, we only need to know how the thing-building thing works. We don't have to understand the resulting product at all. Genetic algorithms are an example - we understand the evolution function, not what evolves from it.
    • mdp2021 2mo ago
      You must have misunderstood the statement: «In order to properly engineer things, we must know how they work»: of course they may work anyway, of course there may be emergent properties (as was explicitly written), but the deontic part of knowledge augmentation, the scientific part, is missing. AI is full of non deterministic devices like genetic algorithms, but there exists a problem of transparency which is paramount in the discipline. Automatically produced solution S works: why does it seem to work? Where would it fail? We must understand that.
      • Borealid 2mo ago
        No, I think you misunderstood me. We engineer the model-training algorithm and then we USE the model. The model is a mess of random gobbledegook that happens to be the best optimizer for the training algorithm's reward function. This is still science, in the same way it's still science when we breed cows to produce more milk without understanding the full bovine genome. If something isn't right with the model we pour extra effort into our science - the training algorithm - and train a new one. At no point does anything ever require peering into the slurry of random digits that make up the model itself, any more than using a computer requires understanding its DRAM training values computed at run time, or any more than pouring water requires understanding the laminar flow along the pitcher's surface (an unsolved math problem!).
• totetsu 2mo ago
  > "what do we mean when we use the word 'reasoning'" This is exactly the most interesting and important point. Because the word reason has been subject to so much philosophical debate throughout history, it is very laden with meanings and implications. It sets all kinds of expectations about normativity, objectivity, truth, and humans convincing other humans. To use the word reason for a stochastic generator is misleading.
  • lanstin 2mo ago
    Consider the difference between “reasonable” and “rational”
• HarHarVeryFunny 2mo ago
  > I don't see a clear demarcation of the things that only "reasoning" can accomplish and can't be approximated or imitated by other methods Human reasoning, and intelligence, of which it is a specialized form, clearly have a biological basis that is being selected for. These aren't just nebulous words - there are some concrete biological/neural mechanisms at work here, and whether or not some "other method(s)" can provide the same, or lesser, or greater, capabilities very much depends on the specifics. There WILL be lines of demarcation and areas of capability overlap and exclusion, but those depend on the specifics, and if the goal is to match or exceed human reasoning capabilities then understanding what those are is obviously going to help. For example if your "other methods" (artificial reasoning) are all varieties of auto-regressive prediction, and the brain is also doing something more that can't be couched as prediction, then you are ngmi!
• thesmtsolver2 2mo ago
  It is not semantics. For decades, logic and CS researchers have known what reasoning is. LLM folks suddenly can’t claim an approximation of that is what constitutes full scale reasoning just because they can achieve only an approximation. Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples but fails on simple multiplications often as it doesn’t have it in its training dataset. We won’t say the program actually multiplies numbers.
  • solid_fuel 2mo ago
    > Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples but fails on simple multiplications often as it doesn’t have it in its training dataset. > We won’t say the program actually multiplies numbers. That's a good analogy. To extend it further, in cases where the calculator can't handle a question - e.g. numbers too large - a properly designed calculator returns an error instead of a randomly hallucinated answer. We haven't even achieved that level of safeguard around token predictors yet.
    • urbsgpw 2mo ago
      But isn't it the case that we can't reach this safeguard with the current architecture? I remember Karpathy making an interesting point 2 years (cca) back, that I would summarize somehow like this: the mechanics behind every LLM answer are the same, what you then call hallucination is more or less a consequence of whether or not the answer was factually correct/useful. Which would mean, as is so often the case, that the "killer feature" of the LLMs is also its biggest weakness and the two can't be disentangled. Now, we are inventive creatures and we might come up with a remedy for these issues, but what you basically see so far is more guardrails, the use of harnesses and building a whole bunch of infrastructure around the LLMs to get useful work out of them. Which, btw is not a critique, I do it as well and it's a fun engineering challenge.
    • simianwords 2mo ago
      This is a strawman. LLMs have the ability to identify when they can’t solve a question just like humans.
• yourapostasy 2mo ago
  Even at the pure functionality endpoint of the spectrum, I find this "wishful mnemonics" aspect very disconcerting. I put my "investor" hat on, and the whole utilitarianism "who cares how or whether it reasons, it's functional let's go" does not forecast the kind of returns to me on a fundamental level. The "it's functional, who cares" black box crowd is leveraging that claim in domains where verification is definitely in P with verifiers like Lean, Coq, or X-ray crystallography. However, in areas like economics, sociology, medicine, or public policy, we lose the luxury of easy verification. If model/harness recommends a complex, counter-intuitive shift in global monetary policy, we cannot simply run a quick polynomial-time checker to see if it works. Verification might require waiting 20 years to see if the economy collapses. If we accept the black box here, we are flying blind. As an investor, I don't see that many deterministic verifiers out there that would justify the forward-projecting valuations beyond the current lofty valuations. Even in mathematics, where verification is easy, there is an epistemological cost. Suppose a model solves the Riemann Hypothesis, but the proof is 50,000 pages of inscrutable statistical associations mapped into logical steps. A computer can verify it is correct, but practically no human mind can comprehend why it is correct. We gain the solution, but we lose the underlying human intuition that usually allows us to connect that solution to other fields of science and knowledge. Then we have "specification gaming." If we rely entirely on an automated verifier to check the model's work, the model will eventually learn to optimize for the flaws in the verifier rather than finding the actual truth. If the verification process has a blind spot, the black box will find it and exploit it, giving us an answer that passes the check but is fundamentally wrong. If we take on above epistemological debt, we might never actually know when our specifications are being gamed, and adopt a solution that is fundamentally incorrect. I've said elsewhere on the Net that I think it is likely we're in a "high grading" era of these models, where we're mining out the "easy" connections. "Easy" here does not mean "light impact". There is a real sense to me of some incredible progress possible with this technology over the next few decades, like real, mass production practical aneutronic fusion or similar civilization-defining advances. I don't want to see this technology misused to lend legitimacy to despotic rule by making oracular pronouncements that are uncritically accepted because early verifier-based pronouncements turned out true in unmistakable ways. The technology is one for the history books, but it has this pernicious effect of highjacking the reasoning centers of many people through their linguistic centers that the Neuro Linguistic Programming crowd could only dream of actually accomplishing.
• raducu 2mo ago
  > I don't see a clear demarcation of the things that only "reasoning" can accomplish. Producing beautiful, coherent, maintainable code. RL trained the models to produce correct code but boy oh boy is it ugly if it's 10k+ LOC